[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:34:21 dist-test-slave-ss4n krb5kdc[10347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:34:21 dist-test-slave-ss4n krb5kdc[10347](info): set up 2 sockets
Aug 06 19:34:21 dist-test-slave-ss4n krb5kdc[10347](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 19:34:23.068367 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.006s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:34:23 dist-test-slave-ss4n krb5kdc[10347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:34:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:34:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:34:23.316555 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:37035
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:44429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:37035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:23.747614 10360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:23.748131 10360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:23.748312 10360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:23.756433 10360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:34:23.756636 10360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:23.756762 10360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:23.756875 10360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:34:23.756968 10360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:34:23.769099 10360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:44429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:37035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:37035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:23.771329 10360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:23.773468 10360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:23.793776 10365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:23.794073 10368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:23.794695 10366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:23.795046 10360 server_base.cc:1034] running on GCE node
I20260806 19:34:23.796485 10360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:23.799214 10360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:23.800537 10360 hybrid_clock.cc:648] HybridClock initialized: now 1786044863800467 us; error 73 us; skew 500 ppm
I20260806 19:34:23.801173 10360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:23 dist-test-slave-ss4n krb5kdc[10347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044863, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:23.805827 10360 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:34:23.809229 10360 webserver.cc:460] Webserver started at http://127.10.25.62:33117/ using document root <none> and password file <none>
I20260806 19:34:23.810751 10360 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:23.810984 10360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:23.811512 10360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:23.816658 10360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "4399687ddb6c4fd288f51fa063e057b9"
format_stamp: "Formatted at 2026-08-06 19:34:23 on dist-test-slave-ss4n"
server_key: "3bd38b29916b5cd38b692e83cfff2cd4"
server_key_iv: "8936b7a5ad8b335d0cbaa9b984819c82"
server_key_version: "encryptionkey@0"
I20260806 19:34:23.817858 10360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "4399687ddb6c4fd288f51fa063e057b9"
format_stamp: "Formatted at 2026-08-06 19:34:23 on dist-test-slave-ss4n"
server_key: "3bd38b29916b5cd38b692e83cfff2cd4"
server_key_iv: "8936b7a5ad8b335d0cbaa9b984819c82"
server_key_version: "encryptionkey@0"
I20260806 19:34:23.827476 10360 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.006s sys 0.001s
I20260806 19:34:23.834926 10375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:23.837852 10360 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:34:23.838271 10360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "4399687ddb6c4fd288f51fa063e057b9"
format_stamp: "Formatted at 2026-08-06 19:34:23 on dist-test-slave-ss4n"
server_key: "3bd38b29916b5cd38b692e83cfff2cd4"
server_key_iv: "8936b7a5ad8b335d0cbaa9b984819c82"
server_key_version: "encryptionkey@0"
I20260806 19:34:23.838563 10360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:23.853397 10360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:23.860325 10360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:23.860817 10360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:23.888664 10360 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:37035
I20260806 19:34:23.888784 10427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:37035 every 8 connection(s)
I20260806 19:34:23.891480 10360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:34:23.895624 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 10360
I20260806 19:34:23.896096 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:34:23.898849 10340 external_mini_cluster.cc:1428] Setting key 11f9a103bb4176f9a14304a9e5d506fe
I20260806 19:34:23.899721 10428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:23.919651 10428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9: Bootstrap starting.
I20260806 19:34:23.928781 10428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:23.931205 10428 log.cc:824] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9: Log is configured to *not* fsync() on all Append() calls
I20260806 19:34:23.937162 10428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9: No bootstrap required, opened a new log
Aug 06 19:34:23 dist-test-slave-ss4n krb5kdc[10347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:23.950994 10428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4399687ddb6c4fd288f51fa063e057b9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37035 } }
I20260806 19:34:23.951601 10428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:23.951848 10428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4399687ddb6c4fd288f51fa063e057b9, State: Initialized, Role: FOLLOWER
I20260806 19:34:23.953181 10428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4399687ddb6c4fd288f51fa063e057b9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37035 } }
I20260806 19:34:23.953683 10428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:34:23.954010 10428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:34:23.954275 10428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:23.957104 10428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4399687ddb6c4fd288f51fa063e057b9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37035 } }
I20260806 19:34:23.958305 10428 leader_election.cc:302] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4399687ddb6c4fd288f51fa063e057b9; no voters:
I20260806 19:34:23.959251 10428 leader_election.cc:288] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:34:23.959627 10433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:34:23.963171 10428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:34:23.963224 10433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [term 1 LEADER]: Becoming Leader. State: Replica: 4399687ddb6c4fd288f51fa063e057b9, State: Running, Role: LEADER
I20260806 19:34:23.964390 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:23.906380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32930 (local address 127.10.25.62:37035)
0806 19:34:23.907674 (+ 1294us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:23.907699 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:23.911407 (+ 3708us) server_negotiation.cc:407] Connection header received
0806 19:34:23.926007 (+ 14600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:23.926097 (+ 90us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:23.926967 (+ 870us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:23.932845 (+ 5878us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:23.933997 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:23.937124 (+ 3127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:23.941360 (+ 4236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:23.942286 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:23.942589 (+ 303us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:23.951003 (+ 8414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:23.951094 (+ 91us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:23.951129 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:23.951209 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:23.955313 (+ 4104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:23.956419 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:23.956427 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:23.956436 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:23.956693 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:23.957242 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:23.957250 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:23.957258 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:23.958277 (+ 1019us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:23.958778 (+ 501us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:23.960219 (+ 1441us) server_negotiation.cc:299] Negotiation successful
0806 19:34:23.960931 (+ 712us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":437,"threads_started":1}
I20260806 19:34:23.964643 10433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4399687ddb6c4fd288f51fa063e057b9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37035 } }
I20260806 19:34:23.976581 10436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4399687ddb6c4fd288f51fa063e057b9. Latest consensus state: current_term: 1 leader_uuid: "4399687ddb6c4fd288f51fa063e057b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4399687ddb6c4fd288f51fa063e057b9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37035 } } }
I20260806 19:34:23.977226 10436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [sys.catalog]: This master's current role is: LEADER
I20260806 19:34:23.977967 10434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4399687ddb6c4fd288f51fa063e057b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4399687ddb6c4fd288f51fa063e057b9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37035 } } }
I20260806 19:34:23.978366 10434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9 [sys.catalog]: This master's current role is: LEADER
W20260806 19:34:23.988934 10449 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:34:23.989194 10449 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:34:23.989384 10450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:34:23.992184 10450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:34:24.007426 10450 catalog_manager.cc:1347] Generated new cluster ID: 53699a83712e40b3a1d9a57a66d08154
I20260806 19:34:24.007616 10450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:34:24.024639 10450 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:34:24.026368 10450 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:34:24.034387 10450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4399687ddb6c4fd288f51fa063e057b9: Generated new TSK 0
I20260806 19:34:24.035631 10450 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:34:24.094638 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:37035
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:44429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:24.514065 10457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:24.514448 10457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:24.514650 10457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:24.523293 10457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:24.523445 10457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:24.523699 10457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:34:24.535058 10457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:44429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:37035
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:24.536621 10457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:24.538331 10457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:24.555431 10464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:24.555393 10465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:24.555358 10467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:24.557242 10457 server_base.cc:1034] running on GCE node
I20260806 19:34:24.558166 10457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:24.559388 10457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:24.560689 10457 hybrid_clock.cc:648] HybridClock initialized: now 1786044864560668 us; error 96 us; skew 500 ppm
I20260806 19:34:24.561089 10457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:24 dist-test-slave-ss4n krb5kdc[10347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044864, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:24.565018 10457 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:34:24.567025 10457 webserver.cc:460] Webserver started at http://127.10.25.1:32917/ using document root <none> and password file <none>
I20260806 19:34:24.567790 10457 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:24.567921 10457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:24.568221 10457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:24.571861 10457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "688420b302e74b91b4c211ca611a5485"
format_stamp: "Formatted at 2026-08-06 19:34:24 on dist-test-slave-ss4n"
server_key: "d56a0b6a362487a4e6dc1330654526e3"
server_key_iv: "de111eae507880004b210d97e112f4e5"
server_key_version: "encryptionkey@0"
I20260806 19:34:24.573101 10457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "688420b302e74b91b4c211ca611a5485"
format_stamp: "Formatted at 2026-08-06 19:34:24 on dist-test-slave-ss4n"
server_key: "d56a0b6a362487a4e6dc1330654526e3"
server_key_iv: "de111eae507880004b210d97e112f4e5"
server_key_version: "encryptionkey@0"
I20260806 19:34:24.579929 10457 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 19:34:24.584362 10475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:24.587505 10457 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 19:34:24.587899 10457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "688420b302e74b91b4c211ca611a5485"
format_stamp: "Formatted at 2026-08-06 19:34:24 on dist-test-slave-ss4n"
server_key: "d56a0b6a362487a4e6dc1330654526e3"
server_key_iv: "de111eae507880004b210d97e112f4e5"
server_key_version: "encryptionkey@0"
I20260806 19:34:24.588173 10457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:24.607776 10457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:24.614326 10457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:24.614701 10457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:24.616065 10457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:34:24.618114 10457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:24.618211 10457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:24.618418 10457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:24.618479 10457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:24.663211 10457 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:45377
I20260806 19:34:24.663275 10602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:45377 every 8 connection(s)
I20260806 19:34:24.664646 10457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:34:24.669242 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 10457
I20260806 19:34:24.669612 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:34:24.670656 10340 external_mini_cluster.cc:1428] Setting key ff4021401c0ead8eccf6391a4f6f0cc9
Aug 06 19:34:24 dist-test-slave-ss4n krb5kdc[10347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044864, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
I20260806 19:34:24.691524 10605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:24.668518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:34711 (local address 127.10.25.62:37035)
0806 19:34:24.669167 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:24.669174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:24.670601 (+ 1427us) server_negotiation.cc:407] Connection header received
0806 19:34:24.673433 (+ 2832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:24.673444 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:24.673646 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:24.673829 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:24.676139 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:24.676970 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:24.678915 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:24.679373 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:24.679532 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:24.684689 (+ 5157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:24.684695 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:24.684698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:24.684742 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:24.687736 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:24.688715 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:24.688720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:24.688722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:24.688878 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:24.689412 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:24.689417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:24.689419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:24.689805 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:24.689978 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:24.690863 (+ 885us) server_negotiation.cc:299] Negotiation successful
0806 19:34:24.691133 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":370,"threads_started":1}
I20260806 19:34:24.692982 10606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:24.669173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:37035 (local address 127.10.25.1:34711)
0806 19:34:24.670367 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:24.670440 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:24.672906 (+ 2466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:24.674388 (+ 1482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:24.674404 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:24.675132 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:24.675839 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:24.675864 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:24.677186 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:24.677191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:24.678711 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:24.678729 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:24.679696 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:24.679705 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:24.679990 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:24.681141 (+ 1151us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:24.681172 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:24.684332 (+ 3160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:24.687949 (+ 3617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:24.687961 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:24.687992 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:24.688537 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:24.689012 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:24.689017 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:24.689021 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:24.689266 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:24.689953 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:24.689963 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:24.690378 (+ 415us) client_negotiation.cc:769] Sending connection context
0806 19:34:24.690764 (+ 386us) client_negotiation.cc:240] Negotiation successful
0806 19:34:24.691398 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":526,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
I20260806 19:34:24.695195 10603 heartbeater.cc:347] Connected to a master server at 127.10.25.62:37035
I20260806 19:34:24.695731 10603 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:24.696762 10603 heartbeater.cc:511] Master 127.10.25.62:37035 requested a full tablet report, sending...
I20260806 19:34:24.702137 10392 ts_manager.cc:194] Registered new tserver with Master: 688420b302e74b91b4c211ca611a5485 (127.10.25.1:45377)
I20260806 19:34:24.704934 10392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:34711
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:34:24.726784 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:37035
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:44429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:25.252851 10609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:25.253218 10609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:25.253397 10609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:25.261737 10609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:25.261955 10609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:25.262267 10609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:34:25.274250 10609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:44429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:37035
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:25.275928 10609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:25.277657 10609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:25.295011 10615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:25.295046 10618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:25.295197 10616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:25.296954 10609 server_base.cc:1034] running on GCE node
I20260806 19:34:25.297997 10609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:25.299331 10609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:25.300773 10609 hybrid_clock.cc:648] HybridClock initialized: now 1786044865300585 us; error 74 us; skew 500 ppm
I20260806 19:34:25.301316 10609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:25 dist-test-slave-ss4n krb5kdc[10347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044865, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:25.306994 10609 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:34:25.309459 10609 webserver.cc:460] Webserver started at http://127.10.25.2:46551/ using document root <none> and password file <none>
I20260806 19:34:25.310416 10609 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:25.310567 10609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:25.310894 10609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:25.314632 10609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "a7ca8c12ba7344c88714fcb8038f778f"
format_stamp: "Formatted at 2026-08-06 19:34:25 on dist-test-slave-ss4n"
server_key: "44ca99e6d57b3c6cd7ae998bd9ade1a8"
server_key_iv: "c4384f9b027bf70991a75b5a2c6fc17e"
server_key_version: "encryptionkey@0"
I20260806 19:34:25.315537 10609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "a7ca8c12ba7344c88714fcb8038f778f"
format_stamp: "Formatted at 2026-08-06 19:34:25 on dist-test-slave-ss4n"
server_key: "44ca99e6d57b3c6cd7ae998bd9ade1a8"
server_key_iv: "c4384f9b027bf70991a75b5a2c6fc17e"
server_key_version: "encryptionkey@0"
I20260806 19:34:25.323958 10609 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260806 19:34:25.328847 10625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:25.330724 10609 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 19:34:25.330922 10609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "a7ca8c12ba7344c88714fcb8038f778f"
format_stamp: "Formatted at 2026-08-06 19:34:25 on dist-test-slave-ss4n"
server_key: "44ca99e6d57b3c6cd7ae998bd9ade1a8"
server_key_iv: "c4384f9b027bf70991a75b5a2c6fc17e"
server_key_version: "encryptionkey@0"
I20260806 19:34:25.331147 10609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:25.367173 10609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:25.372566 10609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:25.372888 10609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:25.374279 10609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:34:25.375891 10609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:25.375989 10609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:25.376101 10609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:25.376149 10609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:25.425535 10609 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:41339
I20260806 19:34:25.425688 10748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:41339 every 8 connection(s)
I20260806 19:34:25.427129 10609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:34:25.435787 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 10609
I20260806 19:34:25.436030 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:34:25.436553 10340 external_mini_cluster.cc:1428] Setting key 6ee0b3ccff511646fd84b3a1f387cb82
Aug 06 19:34:25 dist-test-slave-ss4n krb5kdc[10347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044865, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:25.452893 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:25.430266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:58783 (local address 127.10.25.62:37035)
0806 19:34:25.430776 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:25.430781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:25.432025 (+ 1244us) server_negotiation.cc:407] Connection header received
0806 19:34:25.434309 (+ 2284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:25.434315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:25.434440 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:25.434583 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:25.436685 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:25.437630 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:25.439839 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:25.440280 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:25.440429 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:25.445310 (+ 4881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:25.445317 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:25.445321 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:25.445364 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:25.448751 (+ 3387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:25.449895 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:25.449904 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:25.449913 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:25.450088 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:25.450660 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:25.450665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:25.450667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:25.451002 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:25.451204 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:25.452201 (+ 997us) server_negotiation.cc:299] Negotiation successful
0806 19:34:25.452485 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":269,"threads_started":1}
I20260806 19:34:25.454262 10752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:25.430909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:37035 (local address 127.10.25.2:58783)
0806 19:34:25.431738 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:25.431780 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:25.433954 (+ 2174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:25.434874 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:25.434884 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:25.435557 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:25.436419 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:25.436460 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:25.437807 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:25.437813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:25.439651 (+ 1838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:25.439667 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:25.440541 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:25.440549 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:25.440826 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:25.441676 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:25.441702 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:25.444958 (+ 3256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:25.448995 (+ 4037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:25.449007 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:25.449039 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:25.449735 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:25.450229 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:25.450235 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:25.450239 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:25.450495 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:25.451163 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:25.451173 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:25.451749 (+ 576us) client_negotiation.cc:769] Sending connection context
0806 19:34:25.452089 (+ 340us) client_negotiation.cc:240] Negotiation successful
0806 19:34:25.452724 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":354,"threads_started":1}
I20260806 19:34:25.456243 10749 heartbeater.cc:347] Connected to a master server at 127.10.25.62:37035
I20260806 19:34:25.456734 10749 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:25.458041 10749 heartbeater.cc:511] Master 127.10.25.62:37035 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
I20260806 19:34:25.460456 10392 ts_manager.cc:194] Registered new tserver with Master: a7ca8c12ba7344c88714fcb8038f778f (127.10.25.2:41339)
I20260806 19:34:25.461556 10392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:58783
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:34:25.491709 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:37035
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:44429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:34:25.710227 10603 heartbeater.cc:503] Master 127.10.25.62:37035 was elected leader, sending a full tablet report...
W20260806 19:34:25.979499 10755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:25.980016 10755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:25.980229 10755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:25.992308 10755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:25.992542 10755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:25.992839 10755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:34:26.006835 10755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:44429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:37035
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:26.008424 10755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:26.010057 10755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:26.029119 10761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:26.029232 10760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:26.029100 10763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:26.030165 10755 server_base.cc:1034] running on GCE node
I20260806 19:34:26.030891 10755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:26.032054 10755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:26.033355 10755 hybrid_clock.cc:648] HybridClock initialized: now 1786044866033267 us; error 70 us; skew 500 ppm
I20260806 19:34:26.033893 10755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:26 dist-test-slave-ss4n krb5kdc[10347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044866, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:26.038465 10755 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:34:26.041015 10755 webserver.cc:460] Webserver started at http://127.10.25.3:40839/ using document root <none> and password file <none>
I20260806 19:34:26.042477 10755 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:26.042730 10755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:26.043401 10755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:26.047722 10755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "babd9616f1b04c86b2a47f4ebbb0b90f"
format_stamp: "Formatted at 2026-08-06 19:34:26 on dist-test-slave-ss4n"
server_key: "0053a3c354742f56e8810c7ae852634c"
server_key_iv: "7bc1fbd2b1df9987596727ca259e4cb7"
server_key_version: "encryptionkey@0"
I20260806 19:34:26.048755 10755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "babd9616f1b04c86b2a47f4ebbb0b90f"
format_stamp: "Formatted at 2026-08-06 19:34:26 on dist-test-slave-ss4n"
server_key: "0053a3c354742f56e8810c7ae852634c"
server_key_iv: "7bc1fbd2b1df9987596727ca259e4cb7"
server_key_version: "encryptionkey@0"
I20260806 19:34:26.055964 10755 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260806 19:34:26.061023 10770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:26.063225 10755 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 19:34:26.063483 10755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "babd9616f1b04c86b2a47f4ebbb0b90f"
format_stamp: "Formatted at 2026-08-06 19:34:26 on dist-test-slave-ss4n"
server_key: "0053a3c354742f56e8810c7ae852634c"
server_key_iv: "7bc1fbd2b1df9987596727ca259e4cb7"
server_key_version: "encryptionkey@0"
I20260806 19:34:26.063779 10755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:26.086321 10755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:26.091692 10755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:26.092128 10755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:26.093550 10755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:34:26.095216 10755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:26.095295 10755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:26.095436 10755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:26.095489 10755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:26.147396 10755 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:41967
I20260806 19:34:26.147830 10891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:41967 every 8 connection(s)
I20260806 19:34:26.149021 10755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:34:26.152603 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 10755
I20260806 19:34:26.152869 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:34:26.153708 10340 external_mini_cluster.cc:1428] Setting key 2a7989e97e5e057cc2ab2650c2784966
Aug 06 19:34:26 dist-test-slave-ss4n krb5kdc[10347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044866, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:26.180145 10894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:26.152848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:51911 (local address 127.10.25.62:37035)
0806 19:34:26.154292 (+ 1444us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:26.154304 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:26.155226 (+ 922us) server_negotiation.cc:407] Connection header received
0806 19:34:26.158857 (+ 3631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:26.158862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:26.159020 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:26.159159 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:26.161327 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:26.162271 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:26.164322 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:26.164918 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:26.165081 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:26.170974 (+ 5893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:26.170981 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:26.170985 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:26.171032 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:26.175499 (+ 4467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:26.176669 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:26.176677 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:26.176680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:26.176969 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:26.177861 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:26.177869 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:26.177872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:26.178353 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:26.178539 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:26.179541 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0806 19:34:26.179798 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1221,"thread_start_us":896,"threads_started":1}
I20260806 19:34:26.181241 10895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:26.153532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:37035 (local address 127.10.25.3:51911)
0806 19:34:26.155032 (+ 1500us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:26.155086 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:26.158395 (+ 3309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:26.159465 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:26.159479 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:26.160242 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:26.161086 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:26.161110 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:26.162466 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:26.162470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:26.164125 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:26.164143 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:26.165220 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:26.165230 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:26.165628 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:26.166490 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:26.166545 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:26.170571 (+ 4026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:26.175737 (+ 5166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:26.175746 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:26.175765 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:26.176407 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:26.177196 (+ 789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:26.177201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:26.177204 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:26.177648 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:26.178552 (+ 904us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:26.178560 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:26.178992 (+ 432us) client_negotiation.cc:769] Sending connection context
0806 19:34:26.179370 (+ 378us) client_negotiation.cc:240] Negotiation successful
0806 19:34:26.179962 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1097,"thread_start_us":606,"threads_started":1}
I20260806 19:34:26.183248 10892 heartbeater.cc:347] Connected to a master server at 127.10.25.62:37035
I20260806 19:34:26.183588 10892 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:26.184706 10892 heartbeater.cc:511] Master 127.10.25.62:37035 requested a full tablet report, sending...
I20260806 19:34:26.186754 10392 ts_manager.cc:194] Registered new tserver with Master: babd9616f1b04c86b2a47f4ebbb0b90f (127.10.25.3:41967)
I20260806 19:34:26.188126 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:34:26.188715 10392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:51911
Aug 06 19:34:26 dist-test-slave-ss4n krb5kdc[10347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044866, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 19:34:26 dist-test-slave-ss4n krb5kdc[10347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044866, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:34:26.251005 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:26.228517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51988 (local address 127.10.25.1:45377)
0806 19:34:26.229644 (+ 1127us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:26.229656 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:26.229695 (+ 39us) server_negotiation.cc:407] Connection header received
0806 19:34:26.229775 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:26.229782 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:26.230012 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:26.230193 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:26.232160 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:26.233287 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:26.234836 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:26.236252 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:26.236529 (+ 277us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:26.242439 (+ 5910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:26.242461 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:26.242480 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:26.242543 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:26.246984 (+ 4441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:26.248155 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:26.248168 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:26.248171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:26.248579 (+ 408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:26.249102 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:26.249107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:26.249109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:26.249678 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:26.249957 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:26.250361 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 19:34:26.250644 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":849,"thread_start_us":541,"threads_started":1}
W20260806 19:34:26.252930 10507 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51988
Aug 06 19:34:26 dist-test-slave-ss4n krb5kdc[10347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 19:34:26 dist-test-slave-ss4n krb5kdc[10347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:34:26.308247 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:26.291750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51992 (local address 127.10.25.1:45377)
0806 19:34:26.292012 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:26.292019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:26.292186 (+ 167us) server_negotiation.cc:407] Connection header received
0806 19:34:26.292453 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:26.292457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:26.292529 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:26.292650 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:26.294355 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:26.295162 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:26.296590 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:26.297134 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:26.297296 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:26.302060 (+ 4764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:26.302067 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:26.302071 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:26.302116 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:26.305133 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:26.306098 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:26.306103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:26.306105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:26.306270 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:26.306803 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:26.306808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:26.306810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:26.307150 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:26.307363 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:26.307720 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 19:34:26.307919 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 19:34:26.310602 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 10457
I20260806 19:34:26.329684 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:26.311980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51998 (local address 127.10.25.1:45377)
0806 19:34:26.312510 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:26.312519 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:26.312964 (+ 445us) server_negotiation.cc:407] Connection header received
0806 19:34:26.313207 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:26.313213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:26.313289 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:26.313415 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:26.314943 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:26.315807 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:26.317908 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:26.318477 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:26.318651 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:26.321651 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:26.321657 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:26.321660 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:26.321702 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:26.325721 (+ 4019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:26.326852 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:26.326868 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:26.326870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:26.327072 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:26.327614 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:26.327620 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:26.327626 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:26.327966 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:26.328313 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:26.328803 (+ 490us) server_negotiation.cc:299] Negotiation successful
0806 19:34:26.329120 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260806 19:34:26.330873 10598 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:34:26.429744 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 10457
I20260806 19:34:26.451222 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 10609
Aug 06 19:34:26 dist-test-slave-ss4n krb5kdc[10347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:34:26.464906 10749 heartbeater.cc:503] Master 127.10.25.62:37035 was elected leader, sending a full tablet report...
I20260806 19:34:26.470950 10908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:26.452249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44308 (local address 127.10.25.2:41339)
0806 19:34:26.452916 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:26.452923 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:26.452952 (+ 29us) server_negotiation.cc:407] Connection header received
0806 19:34:26.453024 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:26.453031 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:26.453261 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:26.453478 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:26.454851 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:26.456013 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:26.457453 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:26.458347 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:26.458674 (+ 327us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:26.463605 (+ 4931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:26.463623 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:26.463639 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:26.463685 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:26.467761 (+ 4076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:26.468580 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:26.468588 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:26.468590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:26.468725 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:26.469252 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:26.469263 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:26.469265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:26.469748 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:26.470031 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:26.470369 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 19:34:26.470591 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":324,"threads_started":1}
I20260806 19:34:26.472358 10743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:34:26.564465 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 10609
I20260806 19:34:26.585753 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 10755
Aug 06 19:34:26 dist-test-slave-ss4n krb5kdc[10347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:34:26.607695 10910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:26.587013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34196 (local address 127.10.25.3:41967)
0806 19:34:26.587725 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:26.587733 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:26.587768 (+ 35us) server_negotiation.cc:407] Connection header received
0806 19:34:26.587837 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:26.587844 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:26.588073 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:26.588229 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:26.589749 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:26.591041 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:26.593008 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:26.593976 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:26.594132 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:26.599971 (+ 5839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:26.599991 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:26.600013 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:26.600067 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:26.603679 (+ 3612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:26.604722 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:26.604732 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:26.604737 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:26.604907 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:26.605343 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:26.605348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:26.605352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:26.605922 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:26.606192 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:26.606994 (+ 802us) server_negotiation.cc:299] Negotiation successful
0806 19:34:26.607266 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":377,"threads_started":1}
I20260806 19:34:26.609261 10887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:34:26.700773 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 10755
I20260806 19:34:26.722235 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 10360
I20260806 19:34:26.723886 10423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:34:26.832727 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 10360
2026-08-06T19:34:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4989 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:34:26 dist-test-slave-ss4n krb5kdc[10915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:34:26 dist-test-slave-ss4n krb5kdc[10915](info): set up 2 sockets
Aug 06 19:34:26 dist-test-slave-ss4n krb5kdc[10915](info): commencing operation
krb5kdc: starting...
W20260806 19:34:28.064015 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.001s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:34:28 dist-test-slave-ss4n krb5kdc[10915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:34:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:34:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:34:28.290985 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:34553
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:41009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:34553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:28.684369 10928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:28.684742 10928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:28.684901 10928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:28.693535 10928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:34:28.693674 10928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:28.693760 10928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:28.693854 10928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:34:28.693925 10928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:34:28.704967 10928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:41009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:34553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:34553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:28.706645 10928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:28.708361 10928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:28.723076 10933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:28.723135 10934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:28.723150 10936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:28.724570 10928 server_base.cc:1034] running on GCE node
I20260806 19:34:28.725755 10928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:28.727537 10928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:28.728955 10928 hybrid_clock.cc:648] HybridClock initialized: now 1786044868728844 us; error 56 us; skew 500 ppm
I20260806 19:34:28.729909 10928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:28 dist-test-slave-ss4n krb5kdc[10915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044868, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:28.734861 10928 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:34:28.736836 10928 webserver.cc:460] Webserver started at http://127.10.25.62:45799/ using document root <none> and password file <none>
I20260806 19:34:28.737789 10928 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:28.737922 10928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:28.738312 10928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:28.741977 10928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "5cb3c51e30ee4efd850da3d292217d92"
format_stamp: "Formatted at 2026-08-06 19:34:28 on dist-test-slave-ss4n"
server_key: "87fb81491c0e139c35149bc9d4f6dbcf"
server_key_iv: "85ea32bb4be102752eadacab66131461"
server_key_version: "encryptionkey@0"
I20260806 19:34:28.742944 10928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "5cb3c51e30ee4efd850da3d292217d92"
format_stamp: "Formatted at 2026-08-06 19:34:28 on dist-test-slave-ss4n"
server_key: "87fb81491c0e139c35149bc9d4f6dbcf"
server_key_iv: "85ea32bb4be102752eadacab66131461"
server_key_version: "encryptionkey@0"
I20260806 19:34:28.749665 10928 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 19:34:28.754436 10943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:28.756524 10928 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:34:28.756772 10928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "5cb3c51e30ee4efd850da3d292217d92"
format_stamp: "Formatted at 2026-08-06 19:34:28 on dist-test-slave-ss4n"
server_key: "87fb81491c0e139c35149bc9d4f6dbcf"
server_key_iv: "85ea32bb4be102752eadacab66131461"
server_key_version: "encryptionkey@0"
I20260806 19:34:28.757027 10928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:28.774518 10928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:28.781529 10928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:28.781937 10928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:28.807801 10928 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:34553
I20260806 19:34:28.807852 10999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:34553 every 8 connection(s)
I20260806 19:34:28.809633 10928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:34:28.814996 11000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:28.817399 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 10928
I20260806 19:34:28.817710 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:34:28.818409 10340 external_mini_cluster.cc:1428] Setting key add1ab63362439b61f3eb1e3fedcf1e5
I20260806 19:34:28.828284 11000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92: Bootstrap starting.
I20260806 19:34:28.832149 11000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:28.833582 11000 log.cc:824] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92: Log is configured to *not* fsync() on all Append() calls
Aug 06 19:34:28 dist-test-slave-ss4n krb5kdc[10915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:28.837575 11000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92: No bootstrap required, opened a new log
I20260806 19:34:28.847422 11003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:28.820768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36514 (local address 127.10.25.62:34553)
0806 19:34:28.821856 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:28.821872 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:28.821935 (+ 63us) server_negotiation.cc:407] Connection header received
0806 19:34:28.824016 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:28.824045 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:28.824543 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:28.825049 (+ 506us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:28.826942 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:28.828454 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:28.830084 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:28.830937 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:28.831187 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:28.836598 (+ 5411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:28.836609 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:28.836628 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:28.836679 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:28.840823 (+ 4144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:28.842052 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:28.842061 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:28.842072 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:28.842331 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:28.842871 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:28.842876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:28.842879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:28.843678 (+ 799us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:28.843976 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:28.845050 (+ 1074us) server_negotiation.cc:299] Negotiation successful
0806 19:34:28.846039 (+ 989us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":841,"thread_start_us":462,"threads_started":1}
I20260806 19:34:28.848394 11000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb3c51e30ee4efd850da3d292217d92" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 34553 } }
I20260806 19:34:28.848866 11000 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:28.849009 11000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cb3c51e30ee4efd850da3d292217d92, State: Initialized, Role: FOLLOWER
I20260806 19:34:28.849781 11000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb3c51e30ee4efd850da3d292217d92" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 34553 } }
I20260806 19:34:28.850073 11000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:34:28.850198 11000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:34:28.850353 11000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:28.853066 11000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb3c51e30ee4efd850da3d292217d92" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 34553 } }
I20260806 19:34:28.853747 11000 leader_election.cc:302] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cb3c51e30ee4efd850da3d292217d92; no voters:
I20260806 19:34:28.854569 11000 leader_election.cc:288] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:34:28.854990 11005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:34:28.855865 11005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [term 1 LEADER]: Becoming Leader. State: Replica: 5cb3c51e30ee4efd850da3d292217d92, State: Running, Role: LEADER
I20260806 19:34:28.856489 11005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb3c51e30ee4efd850da3d292217d92" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 34553 } }
I20260806 19:34:28.857507 11000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:34:28.860328 11008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cb3c51e30ee4efd850da3d292217d92. Latest consensus state: current_term: 1 leader_uuid: "5cb3c51e30ee4efd850da3d292217d92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb3c51e30ee4efd850da3d292217d92" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 34553 } } }
I20260806 19:34:28.860350 11007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cb3c51e30ee4efd850da3d292217d92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb3c51e30ee4efd850da3d292217d92" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 34553 } } }
I20260806 19:34:28.860874 11008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [sys.catalog]: This master's current role is: LEADER
I20260806 19:34:28.863173 11007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92 [sys.catalog]: This master's current role is: LEADER
I20260806 19:34:28.863363 11015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:34:28.869810 11015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:34:28.881974 11015 catalog_manager.cc:1347] Generated new cluster ID: 0e9f0eb4260c4c229837c4100ca5681d
I20260806 19:34:28.882118 11015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:34:28.898793 11015 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:34:28.900840 11015 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:34:28.920437 11015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cb3c51e30ee4efd850da3d292217d92: Generated new TSK 0
I20260806 19:34:28.922290 11015 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:34:28.995019 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:34553
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:41009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:29.389778 11030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:29.390126 11030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:29.390231 11030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:29.397917 11030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:29.398202 11030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:29.398356 11030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:34:29.411414 11030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:41009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:34553
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:29.413118 11030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:29.415148 11030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:29.432330 11035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:29.432366 11038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:29.432268 11036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:29.434602 11030 server_base.cc:1034] running on GCE node
I20260806 19:34:29.435410 11030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:29.436689 11030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:29.440209 11030 hybrid_clock.cc:648] HybridClock initialized: now 1786044869440108 us; error 89 us; skew 500 ppm
I20260806 19:34:29.440814 11030 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:29 dist-test-slave-ss4n krb5kdc[10915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044869, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:29.445272 11030 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:34:29.447834 11030 webserver.cc:460] Webserver started at http://127.10.25.1:35391/ using document root <none> and password file <none>
I20260806 19:34:29.448776 11030 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:29.448912 11030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:29.449198 11030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:29.452558 11030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "25a9d00e1c64434896499c63f0ab71c6"
format_stamp: "Formatted at 2026-08-06 19:34:29 on dist-test-slave-ss4n"
server_key: "defffd31e3b3ad0c739333850df7c129"
server_key_iv: "f0f68535d0bea2d41a44a58b50ca78f3"
server_key_version: "encryptionkey@0"
I20260806 19:34:29.454403 11030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "25a9d00e1c64434896499c63f0ab71c6"
format_stamp: "Formatted at 2026-08-06 19:34:29 on dist-test-slave-ss4n"
server_key: "defffd31e3b3ad0c739333850df7c129"
server_key_iv: "f0f68535d0bea2d41a44a58b50ca78f3"
server_key_version: "encryptionkey@0"
I20260806 19:34:29.464193 11030 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.004s
I20260806 19:34:29.470062 11046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:29.472882 11030 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.002s
I20260806 19:34:29.473251 11030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "25a9d00e1c64434896499c63f0ab71c6"
format_stamp: "Formatted at 2026-08-06 19:34:29 on dist-test-slave-ss4n"
server_key: "defffd31e3b3ad0c739333850df7c129"
server_key_iv: "f0f68535d0bea2d41a44a58b50ca78f3"
server_key_version: "encryptionkey@0"
I20260806 19:34:29.473652 11030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:29.493942 11030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:29.501562 11030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:29.502008 11030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:29.503664 11030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:34:29.505419 11030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:29.505586 11030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 19:34:29.505776 11030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:29.505848 11030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:29.557178 11030 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:34369
I20260806 19:34:29.557317 11166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:34369 every 8 connection(s)
I20260806 19:34:29.558947 11030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:34:29.561568 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 11030
I20260806 19:34:29.561832 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:34:29.563215 10340 external_mini_cluster.cc:1428] Setting key f4d5d71bc999872659b919af27ddeb03
Aug 06 19:34:29 dist-test-slave-ss4n krb5kdc[10915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044869, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
I20260806 19:34:29.584225 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:29.562512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:60891 (local address 127.10.25.62:34553)
0806 19:34:29.563314 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:29.563325 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:29.564911 (+ 1586us) server_negotiation.cc:407] Connection header received
0806 19:34:29.566964 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:29.566970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:29.567089 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:29.567240 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:29.569135 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:29.570104 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:29.571983 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:29.572513 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:29.572679 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:29.576881 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:29.576887 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:29.576891 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:29.576932 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:29.580544 (+ 3612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:29.581553 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:29.581558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:29.581560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:29.581700 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:29.582235 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:29.582242 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:29.582245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:29.582543 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:29.582714 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:29.583533 (+ 819us) server_negotiation.cc:299] Negotiation successful
0806 19:34:29.583805 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":370,"threads_started":1}
I20260806 19:34:29.585251 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:29.563206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:34553 (local address 127.10.25.1:60891)
0806 19:34:29.564309 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:29.564368 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:29.566612 (+ 2244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:29.567551 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:29.567562 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:29.568198 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:29.568895 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:29.568922 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:29.570314 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:29.570320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:29.571783 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:29.571800 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:29.572826 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:29.572835 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:29.573142 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:29.574046 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:29.574076 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:29.576568 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:29.580735 (+ 4167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:29.580745 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:29.580771 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:29.581366 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:29.581823 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:29.581828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:29.581832 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:29.582072 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:29.582701 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:29.582709 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:29.583102 (+ 393us) client_negotiation.cc:769] Sending connection context
0806 19:34:29.583419 (+ 317us) client_negotiation.cc:240] Negotiation successful
0806 19:34:29.584038 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":498,"threads_started":1}
I20260806 19:34:29.587173 11167 heartbeater.cc:347] Connected to a master server at 127.10.25.62:34553
I20260806 19:34:29.587476 11167 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:29.588258 11167 heartbeater.cc:511] Master 127.10.25.62:34553 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
I20260806 19:34:29.591943 10962 ts_manager.cc:194] Registered new tserver with Master: 25a9d00e1c64434896499c63f0ab71c6 (127.10.25.1:34369)
I20260806 19:34:29.593819 10962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:60891
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:34:29.629870 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:34553
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:41009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:30.052829 11174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:30.053182 11174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:30.053289 11174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:30.062260 11174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:30.062409 11174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:30.062553 11174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:34:30.074594 11174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:41009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:34553
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:30.076208 11174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:30.079030 11174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:30.101758 11183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:30.101835 11180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:30.101758 11181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:30.102625 11174 server_base.cc:1034] running on GCE node
I20260806 19:34:30.103593 11174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:30.104979 11174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:30.106269 11174 hybrid_clock.cc:648] HybridClock initialized: now 1786044870106212 us; error 52 us; skew 500 ppm
I20260806 19:34:30.106630 11174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:30 dist-test-slave-ss4n krb5kdc[10915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044870, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:30.112028 11174 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:34:30.115183 11174 webserver.cc:460] Webserver started at http://127.10.25.2:36769/ using document root <none> and password file <none>
I20260806 19:34:30.116499 11174 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:30.116716 11174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:30.117079 11174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:30.120826 11174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "96409d53cba64e7296bbca956629da47"
format_stamp: "Formatted at 2026-08-06 19:34:30 on dist-test-slave-ss4n"
server_key: "0971a94885edac148ddd8f6a4a9d2a33"
server_key_iv: "b6482c803c4b6841aa7190ea9f2fac0a"
server_key_version: "encryptionkey@0"
I20260806 19:34:30.121918 11174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "96409d53cba64e7296bbca956629da47"
format_stamp: "Formatted at 2026-08-06 19:34:30 on dist-test-slave-ss4n"
server_key: "0971a94885edac148ddd8f6a4a9d2a33"
server_key_iv: "b6482c803c4b6841aa7190ea9f2fac0a"
server_key_version: "encryptionkey@0"
I20260806 19:34:30.129897 11174 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 19:34:30.134549 11190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:30.137187 11174 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:34:30.137405 11174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "96409d53cba64e7296bbca956629da47"
format_stamp: "Formatted at 2026-08-06 19:34:30 on dist-test-slave-ss4n"
server_key: "0971a94885edac148ddd8f6a4a9d2a33"
server_key_iv: "b6482c803c4b6841aa7190ea9f2fac0a"
server_key_version: "encryptionkey@0"
I20260806 19:34:30.137687 11174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:30.152522 11174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:30.158344 11174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:30.158742 11174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:30.160496 11174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:34:30.162621 11174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:30.163023 11174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:30.163249 11174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:30.163333 11174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:30.213389 11174 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:44253
I20260806 19:34:30.213442 11308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:44253 every 8 connection(s)
I20260806 19:34:30.214849 11174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:34:30.220690 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 11174
I20260806 19:34:30.220947 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:34:30.221453 10340 external_mini_cluster.cc:1428] Setting key 235b8362afc7863ea7f7a54060b70019
Aug 06 19:34:30 dist-test-slave-ss4n krb5kdc[10915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044870, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
I20260806 19:34:30.245065 11313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:30.219527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:55861 (local address 127.10.25.62:34553)
0806 19:34:30.220549 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:30.220565 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:30.221965 (+ 1400us) server_negotiation.cc:407] Connection header received
0806 19:34:30.224038 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:30.224044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:30.224134 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:30.224274 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:30.227130 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:30.228276 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:30.230233 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:30.230718 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:30.230876 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:30.235701 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:30.235707 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:30.235711 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:30.235751 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:30.240332 (+ 4581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:30.241467 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:30.241541 (+ 74us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:30.241545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:30.241718 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:30.242496 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:30.242503 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:30.242506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:30.242925 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:30.243175 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:30.244232 (+ 1057us) server_negotiation.cc:299] Negotiation successful
0806 19:34:30.244558 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":849,"thread_start_us":472,"threads_started":1}
I20260806 19:34:30.246955 11314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:30.220379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:34553 (local address 127.10.25.2:55861)
0806 19:34:30.221723 (+ 1344us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:30.221782 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:30.223736 (+ 1954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:30.224833 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:30.225004 (+ 171us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:30.225984 (+ 980us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:30.226822 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:30.226852 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:30.228522 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:30.228526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:30.230017 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:30.230042 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:30.231014 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:30.231023 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:30.231282 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:30.232155 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:30.232181 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:30.235312 (+ 3131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:30.240607 (+ 5295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:30.240619 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:30.240637 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:30.241268 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:30.241887 (+ 619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:30.241892 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:30.241895 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:30.242136 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:30.243136 (+ 1000us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:30.243147 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:30.243681 (+ 534us) client_negotiation.cc:769] Sending connection context
0806 19:34:30.244082 (+ 401us) client_negotiation.cc:240] Negotiation successful
0806 19:34:30.244776 (+ 694us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"thread_start_us":753,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
I20260806 19:34:30.249423 11309 heartbeater.cc:347] Connected to a master server at 127.10.25.62:34553
I20260806 19:34:30.249899 11309 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:30.250790 11309 heartbeater.cc:511] Master 127.10.25.62:34553 requested a full tablet report, sending...
I20260806 19:34:30.253597 10962 ts_manager.cc:194] Registered new tserver with Master: 96409d53cba64e7296bbca956629da47 (127.10.25.2:44253)
I20260806 19:34:30.254770 10962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:55861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:34:30.289036 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:34553
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:41009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:34:30.598963 11167 heartbeater.cc:503] Master 127.10.25.62:34553 was elected leader, sending a full tablet report...
W20260806 19:34:30.776234 11317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:30.776579 11317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:30.777014 11317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:30.786674 11317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:30.786844 11317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:30.787070 11317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:34:30.799255 11317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:41009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:34553
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:30.800796 11317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:30.802496 11317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:30.823477 11323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:30.823645 11325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:30.823642 11322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:30.825986 11317 server_base.cc:1034] running on GCE node
I20260806 19:34:30.827257 11317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:30.828889 11317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:30.830474 11317 hybrid_clock.cc:648] HybridClock initialized: now 1786044870830604 us; error 262 us; skew 500 ppm
I20260806 19:34:30.831336 11317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:30 dist-test-slave-ss4n krb5kdc[10915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044870, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:30.835883 11317 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:34:30.837987 11317 webserver.cc:460] Webserver started at http://127.10.25.3:44139/ using document root <none> and password file <none>
I20260806 19:34:30.839066 11317 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:30.839247 11317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:30.839752 11317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:30.845127 11317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "8b8410dcd5c241fb8719065dcd14a61f"
format_stamp: "Formatted at 2026-08-06 19:34:30 on dist-test-slave-ss4n"
server_key: "16f4cb35fea7fc6892ca3d90e6573153"
server_key_iv: "3d50f88a1fde4f7f094d1467152b39c9"
server_key_version: "encryptionkey@0"
I20260806 19:34:30.846906 11317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "8b8410dcd5c241fb8719065dcd14a61f"
format_stamp: "Formatted at 2026-08-06 19:34:30 on dist-test-slave-ss4n"
server_key: "16f4cb35fea7fc6892ca3d90e6573153"
server_key_iv: "3d50f88a1fde4f7f094d1467152b39c9"
server_key_version: "encryptionkey@0"
I20260806 19:34:30.854254 11317 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 19:34:30.859383 11332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:30.861816 11317 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:34:30.862051 11317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "8b8410dcd5c241fb8719065dcd14a61f"
format_stamp: "Formatted at 2026-08-06 19:34:30 on dist-test-slave-ss4n"
server_key: "16f4cb35fea7fc6892ca3d90e6573153"
server_key_iv: "3d50f88a1fde4f7f094d1467152b39c9"
server_key_version: "encryptionkey@0"
I20260806 19:34:30.862419 11317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:30.888466 11317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:30.899065 11317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:30.899452 11317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:30.900714 11317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:34:30.902714 11317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:30.902817 11317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:30.902926 11317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:30.902977 11317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:30.990159 11317 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:37509
I20260806 19:34:30.990535 11452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:37509 every 8 connection(s)
I20260806 19:34:30.994078 11317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:34:30.995895 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 11317
I20260806 19:34:30.996294 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:34:30.997417 10340 external_mini_cluster.cc:1428] Setting key 3cdee11fd48dd642b8e017bacc7d1b79
Aug 06 19:34:31 dist-test-slave-ss4n krb5kdc[10915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044870, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:31.030007 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:30.999548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:53419 (local address 127.10.25.62:34553)
0806 19:34:31.000243 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:31.000251 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:31.001797 (+ 1546us) server_negotiation.cc:407] Connection header received
0806 19:34:31.005088 (+ 3291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:31.005095 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:31.005252 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:31.005412 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:31.008711 (+ 3299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.010308 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.012447 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.012904 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.013076 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.019628 (+ 6552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:31.019635 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:31.019639 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:31.019686 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:31.023310 (+ 3624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:31.025016 (+ 1706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:31.025024 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:31.025026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:31.025275 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:31.026247 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:31.026253 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:31.026255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:31.026691 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:31.026917 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:31.028939 (+ 2022us) server_negotiation.cc:299] Negotiation successful
0806 19:34:31.029281 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":316,"threads_started":1}
I20260806 19:34:31.030846 11456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.000382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:34553 (local address 127.10.25.3:53419)
0806 19:34:31.001596 (+ 1214us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:31.001678 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:31.004441 (+ 2763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:31.005855 (+ 1414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:31.005869 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:31.006765 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:31.007982 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:31.008021 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.010716 (+ 2695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.010721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:31.012199 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:31.012218 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.013212 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.013220 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:31.013604 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.014574 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:31.014601 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:31.019287 (+ 4686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:31.023685 (+ 4398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:31.023700 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:31.023724 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:31.024649 (+ 925us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:31.025661 (+ 1012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:31.025665 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:31.025668 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:31.026008 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:31.026984 (+ 976us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:31.026995 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:31.027511 (+ 516us) client_negotiation.cc:769] Sending connection context
0806 19:34:31.027923 (+ 412us) client_negotiation.cc:240] Negotiation successful
0806 19:34:31.028611 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":615,"threads_started":1}
I20260806 19:34:31.033197 11453 heartbeater.cc:347] Connected to a master server at 127.10.25.62:34553
I20260806 19:34:31.033835 11453 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:31.034953 11453 heartbeater.cc:511] Master 127.10.25.62:34553 requested a full tablet report, sending...
I20260806 19:34:31.037600 10962 ts_manager.cc:194] Registered new tserver with Master: 8b8410dcd5c241fb8719065dcd14a61f (127.10.25.3:37509)
I20260806 19:34:31.038924 10962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:53419
I20260806 19:34:31.047465 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:34:31.089356 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.059909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36530 (local address 127.10.25.62:34553)
0806 19:34:31.060446 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:31.060452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:31.061762 (+ 1310us) server_negotiation.cc:407] Connection header received
0806 19:34:31.062059 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:31.062065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:31.062304 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:31.062604 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:31.066727 (+ 4123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.068342 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.071456 (+ 3114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.072230 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.072431 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.076503 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:31.076512 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:31.076516 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:31.076569 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:31.081709 (+ 5140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:31.083089 (+ 1380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:31.083095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:31.083097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:31.083272 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:31.084137 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:31.084143 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:31.084145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:31.085084 (+ 939us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:31.085701 (+ 617us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:31.088403 (+ 2702us) server_negotiation.cc:299] Negotiation successful
0806 19:34:31.088786 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 19:34:31.105274 10962 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:34:31.115597 10962 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:34:31.161235 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.143854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50474 (local address 127.10.25.1:34369)
0806 19:34:31.145054 (+ 1200us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:31.145064 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:31.145803 (+ 739us) server_negotiation.cc:407] Connection header received
0806 19:34:31.146288 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:31.146298 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:31.147209 (+ 911us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:31.147792 (+ 583us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:31.152356 (+ 4564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.153937 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.158422 (+ 4485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.159774 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.159965 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.160243 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:31.160627 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 19:34:31.160846 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":980,"thread_start_us":697,"threads_started":1}
I20260806 19:34:31.161612 11471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.146342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58006 (local address 127.10.25.2:44253)
0806 19:34:31.147950 (+ 1608us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:31.147962 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:31.148897 (+ 935us) server_negotiation.cc:407] Connection header received
0806 19:34:31.149340 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:31.149348 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:31.149815 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:31.150050 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:31.152642 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.154676 (+ 2034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.157980 (+ 3304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.159827 (+ 1847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.159972 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.160287 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:31.161035 (+ 748us) server_negotiation.cc:299] Negotiation successful
0806 19:34:31.161223 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1058,"spinlock_wait_cycles":28032,"thread_start_us":1023,"threads_started":1}
I20260806 19:34:31.161708 11467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.143132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57590 (local address 127.10.25.3:37509)
0806 19:34:31.144325 (+ 1193us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:31.144340 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:31.145671 (+ 1331us) server_negotiation.cc:407] Connection header received
0806 19:34:31.146288 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:31.146298 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:31.147209 (+ 911us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:31.147594 (+ 385us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:31.152339 (+ 4745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.154386 (+ 2047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.158138 (+ 3752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.159827 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.159972 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.160264 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:31.161022 (+ 758us) server_negotiation.cc:299] Negotiation successful
0806 19:34:31.161269 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":964,"thread_start_us":1250,"threads_started":1}
I20260806 19:34:31.161357 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.143331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:34369 (local address 127.0.0.1:50474)
0806 19:34:31.145682 (+ 2351us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:31.145702 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:31.145979 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:31.148469 (+ 2490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:31.148483 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:31.148552 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:31.152011 (+ 3459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:31.152028 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.154113 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.154120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:31.158209 (+ 4089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:31.158221 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.160028 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.160036 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:31.160297 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.160378 (+ 81us) client_negotiation.cc:769] Sending connection context
0806 19:34:31.160554 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 19:34:31.160810 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2168,"mutex_wait_us":1073,"spinlock_wait_cycles":6912,"thread_start_us":638,"threads_started":1}
I20260806 19:34:31.162825 11470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.143165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:37509 (local address 127.0.0.1:57590)
0806 19:34:31.145426 (+ 2261us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:31.145516 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:31.145979 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:31.148486 (+ 2507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:31.148490 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:31.148609 (+ 119us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:31.151966 (+ 3357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:31.151994 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.154689 (+ 2695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.154697 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:31.157831 (+ 3134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:31.157849 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.160483 (+ 2634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.160491 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:31.160743 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.160831 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 19:34:31.161930 (+ 1099us) client_negotiation.cc:240] Negotiation successful
0806 19:34:31.162095 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1719,"mutex_wait_us":311,"spinlock_wait_cycles":11648,"thread_start_us":1920,"threads_started":1}
I20260806 19:34:31.163187 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.147201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:44253 (local address 127.0.0.1:58006)
0806 19:34:31.148746 (+ 1545us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:31.148764 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:31.149133 (+ 369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:31.150422 (+ 1289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:31.150429 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:31.150464 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:31.152388 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:31.152405 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.154870 (+ 2465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.154874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:31.157697 (+ 2823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:31.157716 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.160547 (+ 2831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.160553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:31.160758 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.160835 (+ 77us) client_negotiation.cc:769] Sending connection context
0806 19:34:31.162377 (+ 1542us) client_negotiation.cc:240] Negotiation successful
0806 19:34:31.162527 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1384,"mutex_wait_us":46,"spinlock_wait_cycles":342784,"thread_start_us":1071,"threads_started":1}
I20260806 19:34:31.167685 11242 tablet_service.cc:1467] Processing CreateTablet for tablet 770e9eb121cb46149b8978ac5c3fdf3a (DEFAULT_TABLE table=test-table [id=4647ce7b85ef4664af551a04d725ae96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:31.167490 11097 tablet_service.cc:1467] Processing CreateTablet for tablet 770e9eb121cb46149b8978ac5c3fdf3a (DEFAULT_TABLE table=test-table [id=4647ce7b85ef4664af551a04d725ae96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:31.169350 11242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 770e9eb121cb46149b8978ac5c3fdf3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:31.169391 11097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 770e9eb121cb46149b8978ac5c3fdf3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:31.170461 11385 tablet_service.cc:1467] Processing CreateTablet for tablet 770e9eb121cb46149b8978ac5c3fdf3a (DEFAULT_TABLE table=test-table [id=4647ce7b85ef4664af551a04d725ae96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:31.172356 11385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 770e9eb121cb46149b8978ac5c3fdf3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:31.186004 11473 tablet_bootstrap.cc:492] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47: Bootstrap starting.
I20260806 19:34:31.190028 11474 tablet_bootstrap.cc:492] T 770e9eb121cb46149b8978ac5c3fdf3a P 25a9d00e1c64434896499c63f0ab71c6: Bootstrap starting.
I20260806 19:34:31.192214 11473 tablet_bootstrap.cc:654] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:31.193841 11473 log.cc:824] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47: Log is configured to *not* fsync() on all Append() calls
I20260806 19:34:31.194825 11474 tablet_bootstrap.cc:654] T 770e9eb121cb46149b8978ac5c3fdf3a P 25a9d00e1c64434896499c63f0ab71c6: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:31.196121 11474 log.cc:824] T 770e9eb121cb46149b8978ac5c3fdf3a P 25a9d00e1c64434896499c63f0ab71c6: Log is configured to *not* fsync() on all Append() calls
I20260806 19:34:31.197772 11476 tablet_bootstrap.cc:492] T 770e9eb121cb46149b8978ac5c3fdf3a P 8b8410dcd5c241fb8719065dcd14a61f: Bootstrap starting.
I20260806 19:34:31.200400 11473 tablet_bootstrap.cc:492] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47: No bootstrap required, opened a new log
I20260806 19:34:31.200839 11473 ts_tablet_manager.cc:1397] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.003s
I20260806 19:34:31.203989 11476 tablet_bootstrap.cc:654] T 770e9eb121cb46149b8978ac5c3fdf3a P 8b8410dcd5c241fb8719065dcd14a61f: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:31.205894 11474 tablet_bootstrap.cc:492] T 770e9eb121cb46149b8978ac5c3fdf3a P 25a9d00e1c64434896499c63f0ab71c6: No bootstrap required, opened a new log
I20260806 19:34:31.205996 11476 log.cc:824] T 770e9eb121cb46149b8978ac5c3fdf3a P 8b8410dcd5c241fb8719065dcd14a61f: Log is configured to *not* fsync() on all Append() calls
I20260806 19:34:31.206516 11474 ts_tablet_manager.cc:1397] T 770e9eb121cb46149b8978ac5c3fdf3a P 25a9d00e1c64434896499c63f0ab71c6: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.003s
I20260806 19:34:31.212348 11476 tablet_bootstrap.cc:492] T 770e9eb121cb46149b8978ac5c3fdf3a P 8b8410dcd5c241fb8719065dcd14a61f: No bootstrap required, opened a new log
I20260806 19:34:31.216873 11476 ts_tablet_manager.cc:1397] T 770e9eb121cb46149b8978ac5c3fdf3a P 8b8410dcd5c241fb8719065dcd14a61f: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.004s
I20260806 19:34:31.219493 11473 raft_consensus.cc:348] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b8410dcd5c241fb8719065dcd14a61f" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37509 } } peers { permanent_uuid: "25a9d00e1c64434896499c63f0ab71c6" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34369 } } peers { permanent_uuid: "96409d53cba64e7296bbca956629da47" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44253 } }
I20260806 19:34:31.220570 11473 raft_consensus.cc:374] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:31.220885 11473 raft_consensus.cc:729] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96409d53cba64e7296bbca956629da47, State: Initialized, Role: FOLLOWER
I20260806 19:34:31.222573 11473 consensus_queue.cc:260] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b8410dcd5c241fb8719065dcd14a61f" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37509 } } peers { permanent_uuid: "25a9d00e1c64434896499c63f0ab71c6" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34369 } } peers { permanent_uuid: "96409d53cba64e7296bbca956629da47" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44253 } }
I20260806 19:34:31.227123 11474 raft_consensus.cc:348] T 770e9eb121cb46149b8978ac5c3fdf3a P 25a9d00e1c64434896499c63f0ab71c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b8410dcd5c241fb8719065dcd14a61f" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37509 } } peers { permanent_uuid: "25a9d00e1c64434896499c63f0ab71c6" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34369 } } peers { permanent_uuid: "96409d53cba64e7296bbca956629da47" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44253 } }
I20260806 19:34:31.228053 11474 raft_consensus.cc:374] T 770e9eb121cb46149b8978ac5c3fdf3a P 25a9d00e1c64434896499c63f0ab71c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:31.228346 11474 raft_consensus.cc:729] T 770e9eb121cb46149b8978ac5c3fdf3a P 25a9d00e1c64434896499c63f0ab71c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a9d00e1c64434896499c63f0ab71c6, State: Initialized, Role: FOLLOWER
I20260806 19:34:31.230248 11474 consensus_queue.cc:260] T 770e9eb121cb46149b8978ac5c3fdf3a P 25a9d00e1c64434896499c63f0ab71c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b8410dcd5c241fb8719065dcd14a61f" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37509 } } peers { permanent_uuid: "25a9d00e1c64434896499c63f0ab71c6" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34369 } } peers { permanent_uuid: "96409d53cba64e7296bbca956629da47" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44253 } }
I20260806 19:34:31.232411 11309 heartbeater.cc:503] Master 127.10.25.62:34553 was elected leader, sending a full tablet report...
I20260806 19:34:31.234724 11473 ts_tablet_manager.cc:1428] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47: Time spent starting tablet: real 0.034s user 0.022s sys 0.008s
I20260806 19:34:31.238852 11476 raft_consensus.cc:348] T 770e9eb121cb46149b8978ac5c3fdf3a P 8b8410dcd5c241fb8719065dcd14a61f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b8410dcd5c241fb8719065dcd14a61f" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37509 } } peers { permanent_uuid: "25a9d00e1c64434896499c63f0ab71c6" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34369 } } peers { permanent_uuid: "96409d53cba64e7296bbca956629da47" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44253 } }
I20260806 19:34:31.239880 11476 raft_consensus.cc:374] T 770e9eb121cb46149b8978ac5c3fdf3a P 8b8410dcd5c241fb8719065dcd14a61f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:31.240193 11476 raft_consensus.cc:729] T 770e9eb121cb46149b8978ac5c3fdf3a P 8b8410dcd5c241fb8719065dcd14a61f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b8410dcd5c241fb8719065dcd14a61f, State: Initialized, Role: FOLLOWER
I20260806 19:34:31.241328 11476 consensus_queue.cc:260] T 770e9eb121cb46149b8978ac5c3fdf3a P 8b8410dcd5c241fb8719065dcd14a61f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b8410dcd5c241fb8719065dcd14a61f" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37509 } } peers { permanent_uuid: "25a9d00e1c64434896499c63f0ab71c6" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34369 } } peers { permanent_uuid: "96409d53cba64e7296bbca956629da47" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44253 } }
I20260806 19:34:31.246030 11474 ts_tablet_manager.cc:1428] T 770e9eb121cb46149b8978ac5c3fdf3a P 25a9d00e1c64434896499c63f0ab71c6: Time spent starting tablet: real 0.039s user 0.027s sys 0.005s
I20260806 19:34:31.247298 11453 heartbeater.cc:503] Master 127.10.25.62:34553 was elected leader, sending a full tablet report...
I20260806 19:34:31.248301 11476 ts_tablet_manager.cc:1428] T 770e9eb121cb46149b8978ac5c3fdf3a P 8b8410dcd5c241fb8719065dcd14a61f: Time spent starting tablet: real 0.031s user 0.019s sys 0.011s
W20260806 19:34:31.249958 11454 tablet.cc:2369] T 770e9eb121cb46149b8978ac5c3fdf3a P 8b8410dcd5c241fb8719065dcd14a61f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:34:31.300858 11479 raft_consensus.cc:482] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:34:31.301438 11479 raft_consensus.cc:504] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b8410dcd5c241fb8719065dcd14a61f" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37509 } } peers { permanent_uuid: "25a9d00e1c64434896499c63f0ab71c6" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34369 } } peers { permanent_uuid: "96409d53cba64e7296bbca956629da47" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44253 } }
I20260806 19:34:31.304870 11479 leader_election.cc:288] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b8410dcd5c241fb8719065dcd14a61f (127.10.25.3:37509), 25a9d00e1c64434896499c63f0ab71c6 (127.10.25.1:34369)
W20260806 19:34:31.313961 11168 tablet.cc:2369] T 770e9eb121cb46149b8978ac5c3fdf3a P 25a9d00e1c64434896499c63f0ab71c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:34:31.317399 11467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.305902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:58971 (local address 127.10.25.3:37509)
0806 19:34:31.306386 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:31.306394 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:31.308799 (+ 2405us) server_negotiation.cc:407] Connection header received
0806 19:34:31.309069 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:31.309074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:31.309142 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:31.309259 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:31.310991 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.312018 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.315275 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.316127 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.316309 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.316436 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:31.316841 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 19:34:31.316983 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225}
I20260806 19:34:31.317631 11483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.305972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:37509 (local address 127.10.25.2:58971)
0806 19:34:31.308656 (+ 2684us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:31.308675 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:31.308899 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:31.309276 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:31.309281 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:31.309325 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:31.310693 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:31.310707 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.312156 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.312163 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:31.315000 (+ 2837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:31.315023 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.316404 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.316411 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:31.316643 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.316671 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 19:34:31.316792 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 19:34:31.316957 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2532,"thread_start_us":2014,"threads_started":1}
I20260806 19:34:31.318157 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.305663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:56343 (local address 127.10.25.1:34369)
0806 19:34:31.306034 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:31.306042 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:31.308283 (+ 2241us) server_negotiation.cc:407] Connection header received
0806 19:34:31.308608 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:31.308614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:31.308703 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:31.308822 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:31.310106 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.311441 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.315340 (+ 3899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.316615 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.316856 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.317071 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:31.317739 (+ 668us) server_negotiation.cc:299] Negotiation successful
0806 19:34:31.317916 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260806 19:34:31.318881 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.305180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:34369 (local address 127.10.25.2:56343)
0806 19:34:31.308103 (+ 2923us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:31.308124 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:31.308422 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:31.308904 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:31.308909 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:31.308963 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:31.309904 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:31.309921 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.311626 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.311631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:31.315017 (+ 3386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:31.315039 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.317243 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.317255 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:31.317442 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.317469 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 19:34:31.318377 (+ 908us) client_negotiation.cc:240] Negotiation successful
0806 19:34:31.318530 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2732,"mutex_wait_us":4,"spinlock_wait_cycles":40192,"thread_start_us":2198,"threads_started":1}
I20260806 19:34:31.318799 11407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770e9eb121cb46149b8978ac5c3fdf3a" candidate_uuid: "96409d53cba64e7296bbca956629da47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b8410dcd5c241fb8719065dcd14a61f" is_pre_election: true
I20260806 19:34:31.319684 11407 raft_consensus.cc:2454] T 770e9eb121cb46149b8978ac5c3fdf3a P 8b8410dcd5c241fb8719065dcd14a61f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96409d53cba64e7296bbca956629da47 in term 0.
I20260806 19:34:31.320247 11118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770e9eb121cb46149b8978ac5c3fdf3a" candidate_uuid: "96409d53cba64e7296bbca956629da47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25a9d00e1c64434896499c63f0ab71c6" is_pre_election: true
I20260806 19:34:31.320653 11194 leader_election.cc:302] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b8410dcd5c241fb8719065dcd14a61f, 96409d53cba64e7296bbca956629da47; no voters:
I20260806 19:34:31.320953 11118 raft_consensus.cc:2454] T 770e9eb121cb46149b8978ac5c3fdf3a P 25a9d00e1c64434896499c63f0ab71c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96409d53cba64e7296bbca956629da47 in term 0.
I20260806 19:34:31.321151 11479 raft_consensus.cc:2781] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:34:31.321341 11479 raft_consensus.cc:482] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:34:31.321465 11479 raft_consensus.cc:3037] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:31.324438 11479 raft_consensus.cc:504] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b8410dcd5c241fb8719065dcd14a61f" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37509 } } peers { permanent_uuid: "25a9d00e1c64434896499c63f0ab71c6" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34369 } } peers { permanent_uuid: "96409d53cba64e7296bbca956629da47" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44253 } }
I20260806 19:34:31.325693 11479 leader_election.cc:288] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [CANDIDATE]: Term 1 election: Requested vote from peers 8b8410dcd5c241fb8719065dcd14a61f (127.10.25.3:37509), 25a9d00e1c64434896499c63f0ab71c6 (127.10.25.1:34369)
I20260806 19:34:31.326435 11118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770e9eb121cb46149b8978ac5c3fdf3a" candidate_uuid: "96409d53cba64e7296bbca956629da47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25a9d00e1c64434896499c63f0ab71c6"
I20260806 19:34:31.326305 11407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770e9eb121cb46149b8978ac5c3fdf3a" candidate_uuid: "96409d53cba64e7296bbca956629da47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b8410dcd5c241fb8719065dcd14a61f"
I20260806 19:34:31.326797 11118 raft_consensus.cc:3037] T 770e9eb121cb46149b8978ac5c3fdf3a P 25a9d00e1c64434896499c63f0ab71c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:31.326900 11407 raft_consensus.cc:3037] T 770e9eb121cb46149b8978ac5c3fdf3a P 8b8410dcd5c241fb8719065dcd14a61f [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:31.330767 11407 raft_consensus.cc:2454] T 770e9eb121cb46149b8978ac5c3fdf3a P 8b8410dcd5c241fb8719065dcd14a61f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96409d53cba64e7296bbca956629da47 in term 1.
I20260806 19:34:31.332240 11194 leader_election.cc:302] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b8410dcd5c241fb8719065dcd14a61f, 96409d53cba64e7296bbca956629da47; no voters:
I20260806 19:34:31.332981 11479 raft_consensus.cc:2781] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:34:31.333356 11118 raft_consensus.cc:2454] T 770e9eb121cb46149b8978ac5c3fdf3a P 25a9d00e1c64434896499c63f0ab71c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96409d53cba64e7296bbca956629da47 in term 1.
I20260806 19:34:31.333806 11479 raft_consensus.cc:686] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [term 1 LEADER]: Becoming Leader. State: Replica: 96409d53cba64e7296bbca956629da47, State: Running, Role: LEADER
I20260806 19:34:31.334774 11479 consensus_queue.cc:237] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b8410dcd5c241fb8719065dcd14a61f" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37509 } } peers { permanent_uuid: "25a9d00e1c64434896499c63f0ab71c6" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34369 } } peers { permanent_uuid: "96409d53cba64e7296bbca956629da47" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44253 } }
I20260806 19:34:31.344056 10960 catalog_manager.cc:5495] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96409d53cba64e7296bbca956629da47 (127.10.25.2). New cstate: current_term: 1 leader_uuid: "96409d53cba64e7296bbca956629da47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b8410dcd5c241fb8719065dcd14a61f" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25a9d00e1c64434896499c63f0ab71c6" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96409d53cba64e7296bbca956629da47" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44253 } health_report { overall_health: HEALTHY } } }
Aug 06 19:34:31 dist-test-slave-ss4n krb5kdc[10915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:34:31.425009 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.400619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50478 (local address 127.10.25.1:34369)
0806 19:34:31.401100 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:31.401112 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:31.401885 (+ 773us) server_negotiation.cc:407] Connection header received
0806 19:34:31.402147 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:31.402152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:31.402292 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:31.402447 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:31.404470 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.405364 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.407061 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.407600 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.407780 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.414188 (+ 6408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:31.414209 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:31.414227 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:31.414282 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:31.419255 (+ 4973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:31.421711 (+ 2456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:31.421755 (+ 44us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:31.421758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:31.422073 (+ 315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:31.422806 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:31.422811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:31.422813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:31.423375 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:31.423901 (+ 526us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:31.424340 (+ 439us) server_negotiation.cc:299] Negotiation successful
0806 19:34:31.424569 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
Aug 06 19:34:31 dist-test-slave-ss4n krb5kdc[10915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044871, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 19:34:31.468389 11310 tablet.cc:2369] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47: Can't schedule compaction. Clean time has not been advanced past its initial value.
Password for test-user@KRBTEST.COM:
Aug 06 19:34:31 dist-test-slave-ss4n krb5kdc[10915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044871, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:34:31.508235 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.481996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50494 (local address 127.10.25.1:34369)
0806 19:34:31.482284 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:31.482293 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:31.482535 (+ 242us) server_negotiation.cc:407] Connection header received
0806 19:34:31.482769 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:31.482774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:31.482852 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:31.482967 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:31.486057 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.486998 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.489216 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.490294 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.490873 (+ 579us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.496713 (+ 5840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:31.496720 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:31.496724 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:31.496776 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:31.501957 (+ 5181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:31.504073 (+ 2116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:31.504081 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:31.504084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:31.504323 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:31.505104 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:31.505109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:31.505112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:31.505580 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:31.505950 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:31.507114 (+ 1164us) server_negotiation.cc:299] Negotiation successful
0806 19:34:31.507577 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
Aug 06 19:34:31 dist-test-slave-ss4n krb5kdc[10915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 19:34:31 dist-test-slave-ss4n krb5kdc[10915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:34:31.587018 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.564034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50506 (local address 127.10.25.1:34369)
0806 19:34:31.564441 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:31.564449 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:31.564610 (+ 161us) server_negotiation.cc:407] Connection header received
0806 19:34:31.565195 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:31.565204 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:31.565367 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:31.565504 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:31.568311 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.569805 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.572192 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.572791 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.572936 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.578167 (+ 5231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:31.578173 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:31.578177 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:31.578226 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:31.582763 (+ 4537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:31.583998 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:31.584004 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:31.584006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:31.584217 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:31.585059 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:31.585064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:31.585067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:31.585635 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:31.585945 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:31.586453 (+ 508us) server_negotiation.cc:299] Negotiation successful
0806 19:34:31.586712 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
W20260806 19:34:31.590750 11075 tablet_service.cc:2908] Rejecting scan request for tablet 770e9eb121cb46149b8978ac5c3fdf3a: Uninitialized: clean time has not yet been initialized
I20260806 19:34:31.596561 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 11030
I20260806 19:34:31.619570 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.598764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50514 (local address 127.10.25.1:34369)
0806 19:34:31.599155 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:31.599163 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:31.599482 (+ 319us) server_negotiation.cc:407] Connection header received
0806 19:34:31.599795 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:31.599800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:31.599873 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:31.599992 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:31.601414 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.602330 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.603905 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.604493 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.604681 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.607933 (+ 3252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:31.607940 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:31.607943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:31.607992 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:31.614656 (+ 6664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:31.615757 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:31.615762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:31.615765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:31.615935 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:31.616397 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:31.616403 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:31.616406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:31.617094 (+ 688us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:31.618298 (+ 1204us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:31.618685 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 19:34:31.619021 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260806 19:34:31.620411 11161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:34:31.749236 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 11030
I20260806 19:34:31.749439 11479 consensus_queue.cc:1035] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25a9d00e1c64434896499c63f0ab71c6" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 19:34:31.778764 11192 connection.cc:535] client connection to 127.10.25.1:34369 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 19:34:31.779150 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 11174
W20260806 19:34:31.779417 11192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 19:34:31.781927 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.781061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:34369 (local address 127.10.25.2:39785)
0806 19:34:31.781375 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:31.781716 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:34369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260806 19:34:31.782671 11192 consensus_peers.cc:487] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 -> Peer 25a9d00e1c64434896499c63f0ab71c6 (127.10.25.1:34369): Couldn't send request to peer 25a9d00e1c64434896499c63f0ab71c6. Status: Network error: Client connection negotiation failed: client connection to 127.10.25.1:34369: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 19:34:31.798085 11479 consensus_queue.cc:1035] T 770e9eb121cb46149b8978ac5c3fdf3a P 96409d53cba64e7296bbca956629da47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b8410dcd5c241fb8719065dcd14a61f" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 06 19:34:31 dist-test-slave-ss4n krb5kdc[10915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:34:31.818156 11499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:31.780068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58014 (local address 127.10.25.2:44253)
0806 19:34:31.780877 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:31.780882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:31.780903 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:34:31.780966 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:31.780971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:31.781425 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:31.781664 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:31.784172 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.785247 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.787802 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:31.788703 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:31.789293 (+ 590us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:31.802535 (+ 13242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:31.802554 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:31.802570 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:31.802618 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:31.809782 (+ 7164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:31.811368 (+ 1586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:31.811385 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:31.811393 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:31.811687 (+ 294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:31.813052 (+ 1365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:31.813065 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:31.813071 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:31.814437 (+ 1366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:31.814902 (+ 465us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:31.816938 (+ 2036us) server_negotiation.cc:299] Negotiation successful
0806 19:34:31.817415 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":565,"threads_started":1}
I20260806 19:34:31.820994 11304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:34:31.996881 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 11174
I20260806 19:34:32.031488 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 11317
Aug 06 19:34:32 dist-test-slave-ss4n krb5kdc[10915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:34:32.064853 11505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:32.033339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55800 (local address 127.10.25.3:37509)
0806 19:34:32.034121 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:32.034126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:32.034143 (+ 17us) server_negotiation.cc:407] Connection header received
0806 19:34:32.034208 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:32.034213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:32.034290 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:32.034419 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:32.035956 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:32.036722 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:32.038981 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:32.039555 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:32.039756 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:32.052436 (+ 12680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:32.052459 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:32.052479 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:32.052538 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:32.059050 (+ 6512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:32.060404 (+ 1354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:32.060413 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:32.060415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:32.060629 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:32.062045 (+ 1416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:32.062052 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:32.062056 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:32.062662 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:32.062966 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:32.064006 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0806 19:34:32.064519 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":466,"threads_started":1}
I20260806 19:34:32.066087 11448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:34:32.184101 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 11317
I20260806 19:34:32.207991 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 10928
I20260806 19:34:32.209910 10995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:34:32.360803 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 10928
2026-08-06T19:34:32Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (5534 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:34:32 dist-test-slave-ss4n krb5kdc[11510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:34:32 dist-test-slave-ss4n krb5kdc[11510](info): set up 2 sockets
Aug 06 19:34:32 dist-test-slave-ss4n krb5kdc[11510](info): commencing operation
krb5kdc: starting...
W20260806 19:34:33.602941 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:34:33 dist-test-slave-ss4n krb5kdc[11510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:34:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:34:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:34:33.863624 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:43935
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:43935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:34.344698 11525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:34.345131 11525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:34.345331 11525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:34.354776 11525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:34:34.354995 11525 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 19:34:34.355113 11525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:34.355235 11525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:34.355331 11525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:34:34.355443 11525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:34:34.366688 11525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:43935
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:43935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:34.368222 11525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:34.369746 11525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:34.387904 11533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:34.387885 11531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:34.390937 11530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:34.392560 11525 server_base.cc:1034] running on GCE node
I20260806 19:34:34.393584 11525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:34.395247 11525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:34.396613 11525 hybrid_clock.cc:648] HybridClock initialized: now 1786044874396456 us; error 88 us; skew 500 ppm
I20260806 19:34:34.397403 11525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:34 dist-test-slave-ss4n krb5kdc[11510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044874, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:34.401794 11525 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:34:34.403721 11525 webserver.cc:460] Webserver started at http://127.10.25.62:41323/ using document root <none> and password file <none>
I20260806 19:34:34.404585 11525 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:34.404699 11525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:34.405055 11525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:34.408473 11525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "bdb335339b5543a6b59d50bec57a58aa"
format_stamp: "Formatted at 2026-08-06 19:34:34 on dist-test-slave-ss4n"
server_key: "6298c889ee945786dc72a0a4d26b426d"
server_key_iv: "5eceedce5fc79b20e26a7fca87d69922"
server_key_version: "encryptionkey@0"
I20260806 19:34:34.409412 11525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "bdb335339b5543a6b59d50bec57a58aa"
format_stamp: "Formatted at 2026-08-06 19:34:34 on dist-test-slave-ss4n"
server_key: "6298c889ee945786dc72a0a4d26b426d"
server_key_iv: "5eceedce5fc79b20e26a7fca87d69922"
server_key_version: "encryptionkey@0"
I20260806 19:34:34.417153 11525 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 19:34:34.422066 11540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:34.424645 11525 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260806 19:34:34.424944 11525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "bdb335339b5543a6b59d50bec57a58aa"
format_stamp: "Formatted at 2026-08-06 19:34:34 on dist-test-slave-ss4n"
server_key: "6298c889ee945786dc72a0a4d26b426d"
server_key_iv: "5eceedce5fc79b20e26a7fca87d69922"
server_key_version: "encryptionkey@0"
I20260806 19:34:34.425256 11525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:34.439296 11525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:34.445570 11525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:34.445977 11525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:34.475374 11525 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:43935
I20260806 19:34:34.475396 11607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:43935 every 8 connection(s)
I20260806 19:34:34.477691 11525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:34:34.482529 11608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:34.487082 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 11525
I20260806 19:34:34.487391 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:34:34.488304 10340 external_mini_cluster.cc:1428] Setting key 48b2e2a3c4be7dacf6588a8ef8416847
I20260806 19:34:34.499864 11608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa: Bootstrap starting.
I20260806 19:34:34.504559 11608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:34.506193 11608 log.cc:824] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa: Log is configured to *not* fsync() on all Append() calls
Aug 06 19:34:34 dist-test-slave-ss4n krb5kdc[11510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:34.511541 11608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa: No bootstrap required, opened a new log
I20260806 19:34:34.519109 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:34.491771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33904 (local address 127.10.25.62:43935)
0806 19:34:34.493022 (+ 1251us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:34.493037 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:34.493089 (+ 52us) server_negotiation.cc:407] Connection header received
0806 19:34:34.495019 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:34.495047 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:34.495533 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:34.495946 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:34.498059 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:34.499578 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:34.501574 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:34.502370 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:34.502636 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:34.509447 (+ 6811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:34.509458 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:34.509541 (+ 83us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:34.509616 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:34.513758 (+ 4142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:34.514725 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:34.514734 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:34.514743 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:34.514998 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:34.515468 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:34.515478 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:34.515483 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:34.516329 (+ 846us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:34.516703 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:34.517322 (+ 619us) server_negotiation.cc:299] Negotiation successful
0806 19:34:34.517868 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":975,"thread_start_us":587,"threads_started":1}
I20260806 19:34:34.522536 11608 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb335339b5543a6b59d50bec57a58aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43935 } }
I20260806 19:34:34.523150 11608 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:34.523283 11608 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdb335339b5543a6b59d50bec57a58aa, State: Initialized, Role: FOLLOWER
I20260806 19:34:34.524168 11608 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb335339b5543a6b59d50bec57a58aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43935 } }
I20260806 19:34:34.524585 11608 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:34:34.524747 11608 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:34:34.524964 11608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:34.529161 11608 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb335339b5543a6b59d50bec57a58aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43935 } }
I20260806 19:34:34.529966 11608 leader_election.cc:302] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdb335339b5543a6b59d50bec57a58aa; no voters:
I20260806 19:34:34.530951 11608 leader_election.cc:288] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:34:34.531190 11613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:34:34.533798 11608 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:34:34.533893 11613 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [term 1 LEADER]: Becoming Leader. State: Replica: bdb335339b5543a6b59d50bec57a58aa, State: Running, Role: LEADER
I20260806 19:34:34.534636 11613 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb335339b5543a6b59d50bec57a58aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43935 } }
I20260806 19:34:34.538214 11614 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdb335339b5543a6b59d50bec57a58aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb335339b5543a6b59d50bec57a58aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43935 } } }
I20260806 19:34:34.538878 11614 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [sys.catalog]: This master's current role is: LEADER
I20260806 19:34:34.538483 11615 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdb335339b5543a6b59d50bec57a58aa. Latest consensus state: current_term: 1 leader_uuid: "bdb335339b5543a6b59d50bec57a58aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb335339b5543a6b59d50bec57a58aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43935 } } }
I20260806 19:34:34.539682 11615 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa [sys.catalog]: This master's current role is: LEADER
I20260806 19:34:34.541878 11622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:34:34.547756 11622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:34:34.560016 11622 catalog_manager.cc:1347] Generated new cluster ID: b0384ad20cad4572ab5e3e1cce8cd888
I20260806 19:34:34.560189 11622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:34:34.582202 11622 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:34:34.583829 11622 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:34:34.598544 11622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa: Generated new TSK 0
I20260806 19:34:34.599805 11622 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:34:34.664767 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:43935
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:35.083763 11635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:35.084115 11635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:35.084230 11635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:35.093523 11635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:35.093703 11635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:35.093806 11635 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:34:35.093891 11635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:34:35.105888 11635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.25.62:43935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:35.107369 11635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:35.109129 11635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:35.126158 11640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:35.126158 11643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:35.126190 11641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:35.127676 11635 server_base.cc:1034] running on GCE node
I20260806 19:34:35.128535 11635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:35.130631 11635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:35.132068 11635 hybrid_clock.cc:648] HybridClock initialized: now 1786044875131837 us; error 185 us; skew 500 ppm
I20260806 19:34:35.132473 11635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:35 dist-test-slave-ss4n krb5kdc[11510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044875, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:35.137087 11635 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:34:35.139631 11635 webserver.cc:460] Webserver started at http://127.10.25.1:33077/ using document root <none> and password file <none>
I20260806 19:34:35.140645 11635 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:35.140758 11635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:35.141179 11635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:35.144655 11635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "1ad362f57c5b499daf39a74a3eb892c1"
format_stamp: "Formatted at 2026-08-06 19:34:35 on dist-test-slave-ss4n"
server_key: "7310f5d452a9f75056d938ddab59bccf"
server_key_iv: "23de8cb0ca292f5ccaf282701049c26e"
server_key_version: "encryptionkey@0"
I20260806 19:34:35.145627 11635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "1ad362f57c5b499daf39a74a3eb892c1"
format_stamp: "Formatted at 2026-08-06 19:34:35 on dist-test-slave-ss4n"
server_key: "7310f5d452a9f75056d938ddab59bccf"
server_key_iv: "23de8cb0ca292f5ccaf282701049c26e"
server_key_version: "encryptionkey@0"
I20260806 19:34:35.151758 11635 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 19:34:35.156312 11650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:35.158034 11635 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:34:35.158242 11635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "1ad362f57c5b499daf39a74a3eb892c1"
format_stamp: "Formatted at 2026-08-06 19:34:35 on dist-test-slave-ss4n"
server_key: "7310f5d452a9f75056d938ddab59bccf"
server_key_iv: "23de8cb0ca292f5ccaf282701049c26e"
server_key_version: "encryptionkey@0"
I20260806 19:34:35.158475 11635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:35.183374 11635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:35.188733 11635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:35.189077 11635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:35.192711 11635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:35.192852 11635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:35.193043 11635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:35.193145 11635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:34:35 dist-test-slave-ss4n krb5kdc[11510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044875, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:35.220311 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:35.193538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:37273 (local address 127.10.25.62:43935)
0806 19:34:35.194260 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:35.194268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:35.195535 (+ 1267us) server_negotiation.cc:407] Connection header received
0806 19:34:35.197892 (+ 2357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:35.197903 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:35.198041 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:35.198189 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:35.201184 (+ 2995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:35.202218 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:35.205247 (+ 3029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:35.205927 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:35.206109 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:35.211682 (+ 5573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:35.211688 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:35.211692 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:35.211732 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:35.215723 (+ 3991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:35.216608 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:35.216614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:35.216617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:35.216775 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:35.217291 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:35.217299 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:35.217302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:35.217671 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:35.217827 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:35.219081 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0806 19:34:35.219586 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":436,"threads_started":1}
I20260806 19:34:35.220754 11662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:35.194308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43935 (local address 127.10.25.1:37273)
0806 19:34:35.195305 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:35.195369 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:35.197465 (+ 2096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:35.198490 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:35.198502 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:35.199382 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:35.200894 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:35.200936 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:35.202424 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:35.202428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:35.204972 (+ 2544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:35.205005 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:35.206242 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:35.206251 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:35.206500 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:35.207513 (+ 1013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:35.207556 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:35.211349 (+ 3793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:35.215919 (+ 4570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:35.215936 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:35.215953 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:35.216458 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:35.216920 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:35.216924 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:35.216926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:35.217164 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:35.217827 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:35.217838 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:35.218446 (+ 608us) client_negotiation.cc:769] Sending connection context
0806 19:34:35.218894 (+ 448us) client_negotiation.cc:240] Negotiation successful
0806 19:34:35.219411 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":429,"threads_started":1}
I20260806 19:34:35.244215 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:35.228137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43935 (local address 127.10.25.1:43169)
0806 19:34:35.228945 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:35.228968 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:35.229117 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:35.229627 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:35.229632 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:35.230577 (+ 945us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:35.230929 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:35.230940 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:35.232014 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:35.232026 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:35.233137 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:35.233151 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:35.234276 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:35.234285 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:35.234482 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:35.235268 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:35.235287 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:35.236079 (+ 792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:35.240439 (+ 4360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:35.240444 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:35.240447 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:35.240949 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:35.241457 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:35.241465 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:35.241469 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:35.241668 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:35.242423 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:35.242428 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:35.242785 (+ 357us) client_negotiation.cc:769] Sending connection context
0806 19:34:35.243005 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 19:34:35.243484 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":483,"threads_started":1}
I20260806 19:34:35.245187 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:35.228237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:43169 (local address 127.10.25.62:43935)
0806 19:34:35.228520 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:35.228528 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:35.229084 (+ 556us) server_negotiation.cc:407] Connection header received
0806 19:34:35.229289 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:35.229295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:35.229429 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:35.229591 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:35.231103 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:35.231842 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:35.233300 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:35.234023 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:35.234184 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:35.236328 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:35.236336 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:35.236342 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:35.236394 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:35.239955 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:35.241094 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:35.241101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:35.241106 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:35.241286 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:35.241792 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:35.241800 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:35.241804 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:35.242270 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:35.242466 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:35.244676 (+ 2210us) server_negotiation.cc:299] Negotiation successful
0806 19:34:35.244939 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 19:34:35.255793 11635 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:42401
I20260806 19:34:35.255846 11778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:42401 every 8 connection(s)
I20260806 19:34:35.257393 11635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:34:35.260437 11779 heartbeater.cc:347] Connected to a master server at 127.10.25.62:43935
I20260806 19:34:35.260871 11779 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:35.261803 11779 heartbeater.cc:511] Master 127.10.25.62:43935 requested a full tablet report, sending...
I20260806 19:34:35.264443 11560 ts_manager.cc:194] Registered new tserver with Master: 1ad362f57c5b499daf39a74a3eb892c1 (127.10.25.1:42401)
I20260806 19:34:35.265133 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 11635
I20260806 19:34:35.265393 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:34:35.266251 10340 external_mini_cluster.cc:1428] Setting key 593adffe7883dd7a7cf312f7817396e5
I20260806 19:34:35.266342 11560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:37273
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:34:35.322751 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:43935
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:35.731508 11783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:35.731930 11783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:35.732127 11783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:35.741601 11783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:35.741896 11783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:35.742091 11783 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:34:35.742275 11783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:34:35.756223 11783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.25.62:43935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:35.757948 11783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:35.759799 11783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:35.778033 11791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:35.778124 11789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:35.778153 11788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:35.778990 11783 server_base.cc:1034] running on GCE node
I20260806 19:34:35.779774 11783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:35.781014 11783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:35.782290 11783 hybrid_clock.cc:648] HybridClock initialized: now 1786044875782205 us; error 67 us; skew 500 ppm
I20260806 19:34:35.782655 11783 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:35 dist-test-slave-ss4n krb5kdc[11510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044875, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:35.786880 11783 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:34:35.788829 11783 webserver.cc:460] Webserver started at http://127.10.25.2:38491/ using document root <none> and password file <none>
I20260806 19:34:35.789742 11783 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:35.789857 11783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:35.790192 11783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:35.793855 11783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "7a1647c26a87478f915d648281ce4957"
format_stamp: "Formatted at 2026-08-06 19:34:35 on dist-test-slave-ss4n"
server_key: "b23b710a88b1ed26695a29cb381817d7"
server_key_iv: "177913daa07a0b45d432be222f4cfd63"
server_key_version: "encryptionkey@0"
I20260806 19:34:35.794819 11783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "7a1647c26a87478f915d648281ce4957"
format_stamp: "Formatted at 2026-08-06 19:34:35 on dist-test-slave-ss4n"
server_key: "b23b710a88b1ed26695a29cb381817d7"
server_key_iv: "177913daa07a0b45d432be222f4cfd63"
server_key_version: "encryptionkey@0"
I20260806 19:34:35.801390 11783 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:34:35.805905 11798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:35.807701 11783 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:34:35.807884 11783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "7a1647c26a87478f915d648281ce4957"
format_stamp: "Formatted at 2026-08-06 19:34:35 on dist-test-slave-ss4n"
server_key: "b23b710a88b1ed26695a29cb381817d7"
server_key_iv: "177913daa07a0b45d432be222f4cfd63"
server_key_version: "encryptionkey@0"
I20260806 19:34:35.808120 11783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:35.828665 11783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:35.835363 11783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:35.835682 11783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:35.840010 11783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:35.840183 11783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:35.840430 11783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:35.840519 11783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:34:35 dist-test-slave-ss4n krb5kdc[11510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044875, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:35.876971 11807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:35.840472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:47191 (local address 127.10.25.62:43935)
0806 19:34:35.841191 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:35.841199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:35.842897 (+ 1698us) server_negotiation.cc:407] Connection header received
0806 19:34:35.846638 (+ 3741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:35.846650 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:35.846811 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:35.847025 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:35.850442 (+ 3417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:35.851824 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:35.855580 (+ 3756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:35.856090 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:35.856267 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:35.863591 (+ 7324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:35.863604 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:35.863614 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:35.863688 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:35.869460 (+ 5772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:35.872236 (+ 2776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:35.872248 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:35.872254 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:35.872552 (+ 298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:35.873678 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:35.873685 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:35.873734 (+ 49us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:35.874197 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:35.874394 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:35.875984 (+ 1590us) server_negotiation.cc:299] Negotiation successful
0806 19:34:35.876385 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":270,"threads_started":1}
I20260806 19:34:35.881546 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:35.841370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43935 (local address 127.10.25.2:47191)
0806 19:34:35.842632 (+ 1262us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:35.842704 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:35.846079 (+ 3375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:35.847445 (+ 1366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:35.847463 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:35.848661 (+ 1198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:35.849987 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:35.850028 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:35.852172 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:35.852183 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:35.855244 (+ 3061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:35.855286 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:35.856415 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:35.856426 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:35.856751 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:35.857787 (+ 1036us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:35.857835 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:35.863036 (+ 5201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:35.869991 (+ 6955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:35.870013 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:35.870051 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:35.871834 (+ 1783us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:35.872822 (+ 988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:35.872833 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:35.872841 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:35.873370 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:35.874409 (+ 1039us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:35.874424 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:35.875197 (+ 773us) client_negotiation.cc:769] Sending connection context
0806 19:34:35.877315 (+ 2118us) client_negotiation.cc:240] Negotiation successful
0806 19:34:35.878452 (+ 1137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"thread_start_us":443,"threads_started":1}
I20260806 19:34:35.902936 11783 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:46771
I20260806 19:34:35.902937 11927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:46771 every 8 connection(s)
I20260806 19:34:35.904424 11783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:34:35.907675 11928 heartbeater.cc:347] Connected to a master server at 127.10.25.62:43935
I20260806 19:34:35.908002 11928 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:35.908932 11928 heartbeater.cc:511] Master 127.10.25.62:43935 requested a full tablet report, sending...
I20260806 19:34:35.911461 11560 ts_manager.cc:194] Registered new tserver with Master: 7a1647c26a87478f915d648281ce4957 (127.10.25.2:46771)
I20260806 19:34:35.912393 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 11783
I20260806 19:34:35.912648 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:34:35.913244 10340 external_mini_cluster.cc:1428] Setting key 98115b20a29bc70c437003e112323dfd
I20260806 19:34:35.913175 11560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:47191
I20260806 19:34:35.913470 11807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:35.893287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:48059 (local address 127.10.25.62:43935)
0806 19:34:35.893661 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:35.893672 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:35.894910 (+ 1238us) server_negotiation.cc:407] Connection header received
0806 19:34:35.895121 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:35.895128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:35.895233 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:35.895377 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:35.896538 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:35.897345 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:35.900102 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:35.900689 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:35.900875 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:35.903756 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:35.903763 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:35.903766 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:35.903807 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:35.908216 (+ 4409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:35.909727 (+ 1511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:35.909733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:35.909737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:35.909940 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:35.910769 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:35.910776 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:35.910780 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:35.911482 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:35.911762 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:35.912721 (+ 959us) server_negotiation.cc:299] Negotiation successful
0806 19:34:35.912998 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260806 19:34:35.919041 11916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:35.893257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43935 (local address 127.10.25.2:48059)
0806 19:34:35.894630 (+ 1373us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:35.894665 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:35.894945 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:35.895430 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:35.895436 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:35.895962 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:35.896328 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:35.896340 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:35.897711 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:35.897716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:35.899841 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:35.899866 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:35.900966 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:35.900974 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:35.901189 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:35.902031 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:35.902052 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:35.903463 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:35.908551 (+ 5088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:35.908558 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:35.908562 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:35.909379 (+ 817us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:35.910172 (+ 793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:35.910180 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:35.910184 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:35.910548 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:35.911766 (+ 1218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:35.911773 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:35.912026 (+ 253us) client_negotiation.cc:769] Sending connection context
0806 19:34:35.912490 (+ 464us) client_negotiation.cc:240] Negotiation successful
0806 19:34:35.912926 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1093,"thread_start_us":850,"threads_started":1}
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:34:35.977967 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:43935
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:34:36.270093 11779 heartbeater.cc:503] Master 127.10.25.62:43935 was elected leader, sending a full tablet report...
W20260806 19:34:36.389931 11932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:36.390527 11932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:36.390762 11932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:36.400645 11932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:36.400844 11932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:36.401043 11932 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:34:36.401186 11932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:34:36.413895 11932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.25.62:43935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:36.415951 11932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:36.418299 11932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:36.436445 11938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:36.436483 11937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:36.436532 11940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:36.438897 11932 server_base.cc:1034] running on GCE node
I20260806 19:34:36.440162 11932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:36.441685 11932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:36.443022 11932 hybrid_clock.cc:648] HybridClock initialized: now 1786044876442920 us; error 83 us; skew 500 ppm
I20260806 19:34:36.443612 11932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:36 dist-test-slave-ss4n krb5kdc[11510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044876, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:36.447448 11932 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:34:36.449283 11932 webserver.cc:460] Webserver started at http://127.10.25.3:45699/ using document root <none> and password file <none>
I20260806 19:34:36.450145 11932 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:36.450248 11932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:36.450524 11932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:36.453930 11932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "c125e721bd814a608f9ae49d2667edf3"
format_stamp: "Formatted at 2026-08-06 19:34:36 on dist-test-slave-ss4n"
server_key: "6dc1fb481f788b5f5bbd464ba4dc937e"
server_key_iv: "7649b13992d3532ec0e224aed50cd0f7"
server_key_version: "encryptionkey@0"
I20260806 19:34:36.454851 11932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "c125e721bd814a608f9ae49d2667edf3"
format_stamp: "Formatted at 2026-08-06 19:34:36 on dist-test-slave-ss4n"
server_key: "6dc1fb481f788b5f5bbd464ba4dc937e"
server_key_iv: "7649b13992d3532ec0e224aed50cd0f7"
server_key_version: "encryptionkey@0"
I20260806 19:34:36.461565 11932 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:34:36.467465 11947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:36.469780 11932 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 19:34:36.470041 11932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "c125e721bd814a608f9ae49d2667edf3"
format_stamp: "Formatted at 2026-08-06 19:34:36 on dist-test-slave-ss4n"
server_key: "6dc1fb481f788b5f5bbd464ba4dc937e"
server_key_iv: "7649b13992d3532ec0e224aed50cd0f7"
server_key_version: "encryptionkey@0"
I20260806 19:34:36.470320 11932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:36.486683 11932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:36.494582 11932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:36.495018 11932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:36.499727 11932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:36.499884 11932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:36.500073 11932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:36.500186 11932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:34:36 dist-test-slave-ss4n krb5kdc[11510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044876, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:36.536679 11957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:36.501270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43935 (local address 127.10.25.3:54927)
0806 19:34:36.502617 (+ 1347us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:36.502681 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:36.505760 (+ 3079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:36.507248 (+ 1488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:36.507262 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:36.508749 (+ 1487us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:36.510056 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:36.510132 (+ 76us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.512097 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.512106 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:36.516218 (+ 4112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:36.516268 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.517921 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.517931 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:36.518215 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:36.519289 (+ 1074us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:36.519359 (+ 70us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:36.524252 (+ 4893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:36.529888 (+ 5636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:36.529923 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:36.529955 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:36.530891 (+ 936us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:36.531604 (+ 713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:36.531608 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:36.531611 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:36.531866 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:36.532782 (+ 916us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:36.532788 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:36.533546 (+ 758us) client_negotiation.cc:769] Sending connection context
0806 19:34:36.534073 (+ 527us) client_negotiation.cc:240] Negotiation successful
0806 19:34:36.534847 (+ 774us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1018,"thread_start_us":704,"threads_started":1}
I20260806 19:34:36.538090 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:36.500471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:54927 (local address 127.10.25.62:43935)
0806 19:34:36.501436 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:36.501440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:36.502882 (+ 1442us) server_negotiation.cc:407] Connection header received
0806 19:34:36.506460 (+ 3578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:36.506469 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:36.506619 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:36.506797 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:36.510550 (+ 3753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.511780 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.516754 (+ 4974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.517399 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.517655 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:36.524633 (+ 6978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:36.524639 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:36.524643 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:36.524687 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:36.529533 (+ 4846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:36.531100 (+ 1567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:36.531111 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:36.531117 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:36.531402 (+ 285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:36.532037 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:36.532042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:36.532046 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:36.532542 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:36.532712 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:36.537299 (+ 4587us) server_negotiation.cc:299] Negotiation successful
0806 19:34:36.537635 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":721,"threads_started":1}
I20260806 19:34:36.572415 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:36.545437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:57921 (local address 127.10.25.62:43935)
0806 19:34:36.545725 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:36.545732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:36.546891 (+ 1159us) server_negotiation.cc:407] Connection header received
0806 19:34:36.547263 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:36.547268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:36.547345 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:36.547473 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:36.549915 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.551405 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.554778 (+ 3373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.556039 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.556400 (+ 361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:36.559523 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:36.559534 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:36.559542 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:36.559599 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:36.566103 (+ 6504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:36.568019 (+ 1916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:36.568027 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:36.568031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:36.568311 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:36.568973 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:36.568982 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:36.568989 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:36.569642 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:36.569863 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:36.571089 (+ 1226us) server_negotiation.cc:299] Negotiation successful
0806 19:34:36.571566 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260806 19:34:36.573575 12013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:36.545461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43935 (local address 127.10.25.3:57921)
0806 19:34:36.546727 (+ 1266us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:36.546759 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:36.547038 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:36.547500 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:36.547506 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:36.548432 (+ 926us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:36.549547 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:36.549569 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.551622 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.551628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:36.554326 (+ 2698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:36.554351 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.556559 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.556575 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:36.556950 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:36.557937 (+ 987us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:36.557979 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:36.559175 (+ 1196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:36.566442 (+ 7267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:36.566449 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:36.566453 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:36.567707 (+ 1254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:36.568505 (+ 798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:36.568518 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:36.568523 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:36.568771 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:36.569978 (+ 1207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:36.569986 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:36.570467 (+ 481us) client_negotiation.cc:769] Sending connection context
0806 19:34:36.570790 (+ 323us) client_negotiation.cc:240] Negotiation successful
0806 19:34:36.571688 (+ 898us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1075,"thread_start_us":834,"threads_started":1}
I20260806 19:34:36.581929 11932 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:40035
I20260806 19:34:36.581971 12073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:40035 every 8 connection(s)
I20260806 19:34:36.583940 11932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:34:36.588196 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 11932
I20260806 19:34:36.588455 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:34:36.588521 12074 heartbeater.cc:347] Connected to a master server at 127.10.25.62:43935
I20260806 19:34:36.588886 12074 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:36.589021 10340 external_mini_cluster.cc:1428] Setting key 47ebd1623552a17571976c618ef6b954
I20260806 19:34:36.590014 12074 heartbeater.cc:511] Master 127.10.25.62:43935 requested a full tablet report, sending...
I20260806 19:34:36.592180 11560 ts_manager.cc:194] Registered new tserver with Master: c125e721bd814a608f9ae49d2667edf3 (127.10.25.3:40035)
I20260806 19:34:36.593421 11560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:54927
I20260806 19:34:36.593740 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 19:34:36 dist-test-slave-ss4n krb5kdc[11510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044876, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 19:34:36 dist-test-slave-ss4n krb5kdc[11510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044876, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:36.657349 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:36.638353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33916 (local address 127.10.25.62:43935)
0806 19:34:36.638683 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:36.638690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:36.638858 (+ 168us) server_negotiation.cc:407] Connection header received
0806 19:34:36.639104 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:36.639108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:36.639207 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:36.639324 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:36.641234 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.642188 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.643621 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.644071 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.644226 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:36.649124 (+ 4898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:36.649133 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:36.649138 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:36.649209 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:36.653035 (+ 3826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:36.654473 (+ 1438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:36.654483 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:36.654486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:36.654722 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:36.655466 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:36.655475 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:36.655478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:36.655969 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:36.656261 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:36.656687 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 19:34:36.657010 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 19:34:36.663376 11560 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:34:36.672796 11560 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:34:36.717175 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:36.691707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59450 (local address 127.10.25.1:42401)
0806 19:34:36.705371 (+ 13664us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:36.705387 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:36.706273 (+ 886us) server_negotiation.cc:407] Connection header received
0806 19:34:36.706637 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:36.706648 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:36.707256 (+ 608us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:36.707507 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:36.708342 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.710139 (+ 1797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.713467 (+ 3328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.715557 (+ 2090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.715745 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:36.716031 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:36.716621 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 19:34:36.716811 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13373,"thread_start_us":14297,"threads_started":1}
I20260806 19:34:36.717200 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:36.691431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:46771 (local address 127.0.0.1:53618)
0806 19:34:36.706716 (+ 15285us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:36.706752 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:36.707129 (+ 377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:36.707849 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:36.707854 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:36.707878 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:36.708358 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:36.708367 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.710338 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.710345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:36.713581 (+ 3236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:36.713600 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.715889 (+ 2289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.715897 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:36.716274 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:36.716320 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 19:34:36.716418 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 19:34:36.716692 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15032,"mutex_wait_us":149,"thread_start_us":14687,"threads_started":1}
I20260806 19:34:36.717236 12090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:36.691995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53618 (local address 127.10.25.2:46771)
0806 19:34:36.705958 (+ 13963us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:36.705969 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:36.707208 (+ 1239us) server_negotiation.cc:407] Connection header received
0806 19:34:36.707341 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:36.707349 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:36.707642 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:36.707864 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:36.708627 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.710171 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.713827 (+ 3656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.715589 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.716290 (+ 701us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:36.716596 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:36.716701 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 19:34:36.716916 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13798,"thread_start_us":13629,"threads_started":1}
I20260806 19:34:36.718317 12088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:36.691399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:42401 (local address 127.0.0.1:59450)
0806 19:34:36.705514 (+ 14115us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:36.705604 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:36.705943 (+ 339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:36.707527 (+ 1584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:36.707536 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:36.707568 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:36.708069 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:36.708083 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.710350 (+ 2267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.710355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:36.713220 (+ 2865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:36.713241 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.715827 (+ 2586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.715835 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:36.716148 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:36.716242 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 19:34:36.716432 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 19:34:36.717908 (+ 1476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13646,"mutex_wait_us":1404,"thread_start_us":14847,"threads_started":1}
I20260806 19:34:36.720572 12091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:36.708208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42930 (local address 127.10.25.3:40035)
0806 19:34:36.709185 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:36.709194 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:36.710289 (+ 1095us) server_negotiation.cc:407] Connection header received
0806 19:34:36.710635 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:36.710646 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:36.711079 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:36.711311 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:36.712716 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.714426 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.717242 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.718960 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.719073 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:36.719318 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:36.719982 (+ 664us) server_negotiation.cc:299] Negotiation successful
0806 19:34:36.720175 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":475,"threads_started":1}
I20260806 19:34:36.721266 12092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:36.709116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:40035 (local address 127.0.0.1:42930)
0806 19:34:36.710149 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:36.710171 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:36.710407 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:36.711790 (+ 1383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:36.711795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:36.711830 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:36.712433 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:36.712449 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.714591 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.714596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:36.717007 (+ 2411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:36.717024 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:36.719524 (+ 2500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:36.719530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:36.719743 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:36.719779 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 19:34:36.720799 (+ 1020us) client_negotiation.cc:240] Negotiation successful
0806 19:34:36.720966 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"spinlock_wait_cycles":701696,"thread_start_us":315,"threads_started":1}
I20260806 19:34:36.722273 11851 tablet_service.cc:1467] Processing CreateTablet for tablet cbba09874fbd4945a277479a7c4265a7 (DEFAULT_TABLE table=test-table [id=fe0c6b7cd0404e03bdb986f675af6a65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:36.722676 11705 tablet_service.cc:1467] Processing CreateTablet for tablet cbba09874fbd4945a277479a7c4265a7 (DEFAULT_TABLE table=test-table [id=fe0c6b7cd0404e03bdb986f675af6a65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:36.724004 11851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbba09874fbd4945a277479a7c4265a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:36.724097 11705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbba09874fbd4945a277479a7c4265a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:36.725026 11999 tablet_service.cc:1467] Processing CreateTablet for tablet cbba09874fbd4945a277479a7c4265a7 (DEFAULT_TABLE table=test-table [id=fe0c6b7cd0404e03bdb986f675af6a65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:36.726629 11999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbba09874fbd4945a277479a7c4265a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:36.742522 12094 tablet_bootstrap.cc:492] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1: Bootstrap starting.
I20260806 19:34:36.743805 12093 tablet_bootstrap.cc:492] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957: Bootstrap starting.
I20260806 19:34:36.744840 12095 tablet_bootstrap.cc:492] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3: Bootstrap starting.
I20260806 19:34:36.748382 12095 tablet_bootstrap.cc:654] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:36.748456 12094 tablet_bootstrap.cc:654] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:36.750216 12095 log.cc:824] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3: Log is configured to *not* fsync() on all Append() calls
I20260806 19:34:36.750933 12094 log.cc:824] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1: Log is configured to *not* fsync() on all Append() calls
I20260806 19:34:36.753641 12093 tablet_bootstrap.cc:654] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:36.755517 12095 tablet_bootstrap.cc:492] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3: No bootstrap required, opened a new log
I20260806 19:34:36.755767 12093 log.cc:824] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957: Log is configured to *not* fsync() on all Append() calls
I20260806 19:34:36.756017 12095 ts_tablet_manager.cc:1397] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20260806 19:34:36.757794 12094 tablet_bootstrap.cc:492] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1: No bootstrap required, opened a new log
I20260806 19:34:36.758235 12094 ts_tablet_manager.cc:1397] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.005s
I20260806 19:34:36.760953 12093 tablet_bootstrap.cc:492] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957: No bootstrap required, opened a new log
I20260806 19:34:36.761519 12093 ts_tablet_manager.cc:1397] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.000s
I20260806 19:34:36.770164 12095 raft_consensus.cc:348] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:36.770737 12095 raft_consensus.cc:374] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:36.770840 12095 raft_consensus.cc:729] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c125e721bd814a608f9ae49d2667edf3, State: Initialized, Role: FOLLOWER
I20260806 19:34:36.771620 12095 consensus_queue.cc:260] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:36.772967 12074 heartbeater.cc:503] Master 127.10.25.62:43935 was elected leader, sending a full tablet report...
I20260806 19:34:36.774245 12095 ts_tablet_manager.cc:1428] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3: Time spent starting tablet: real 0.018s user 0.019s sys 0.000s
I20260806 19:34:36.778147 12093 raft_consensus.cc:348] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:36.778149 12094 raft_consensus.cc:348] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:36.778714 12093 raft_consensus.cc:374] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:36.778723 12094 raft_consensus.cc:374] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:36.778841 12094 raft_consensus.cc:729] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ad362f57c5b499daf39a74a3eb892c1, State: Initialized, Role: FOLLOWER
I20260806 19:34:36.778841 12093 raft_consensus.cc:729] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a1647c26a87478f915d648281ce4957, State: Initialized, Role: FOLLOWER
I20260806 19:34:36.779605 12093 consensus_queue.cc:260] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:36.779605 12094 consensus_queue.cc:260] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:36.783520 11928 heartbeater.cc:503] Master 127.10.25.62:43935 was elected leader, sending a full tablet report...
I20260806 19:34:36.784181 12094 ts_tablet_manager.cc:1428] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1: Time spent starting tablet: real 0.026s user 0.018s sys 0.006s
I20260806 19:34:36.784153 12093 ts_tablet_manager.cc:1428] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957: Time spent starting tablet: real 0.022s user 0.011s sys 0.010s
W20260806 19:34:36.837648 12075 tablet.cc:2369] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:34:36.907972 11929 tablet.cc:2369] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:34:37.009865 12101 raft_consensus.cc:482] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:34:37.010298 12101 raft_consensus.cc:504] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
W20260806 19:34:37.011127 11780 tablet.cc:2369] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:34:37.012660 12101 leader_election.cc:288] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ad362f57c5b499daf39a74a3eb892c1 (127.10.25.1:42401), c125e721bd814a608f9ae49d2667edf3 (127.10.25.3:40035)
I20260806 19:34:37.021224 12091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.014203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:59079 (local address 127.10.25.3:40035)
0806 19:34:37.014405 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:37.014411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:37.014754 (+ 343us) server_negotiation.cc:407] Connection header received
0806 19:34:37.014918 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:37.014923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:37.015012 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:37.015142 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:37.015871 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.016888 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.019086 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.019977 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.020153 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.020310 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:37.020721 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 19:34:37.020888 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 19:34:37.021220 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.013255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:47097 (local address 127.10.25.1:42401)
0806 19:34:37.013453 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:37.013528 (+ 75us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:37.014095 (+ 567us) server_negotiation.cc:407] Connection header received
0806 19:34:37.014452 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:37.014461 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:37.014565 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:37.014634 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:37.015673 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.016501 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.019071 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.019983 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.020196 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.020327 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:37.020708 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 19:34:37.020891 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 19:34:37.021445 12103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.013909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:40035 (local address 127.10.25.2:59079)
0806 19:34:37.014586 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:37.014600 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:37.014779 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:37.015197 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:37.015200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:37.015222 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:37.015691 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.015703 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.017073 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.017078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.018835 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.018850 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.020275 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.020282 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.020499 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.020528 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 19:34:37.020669 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 19:34:37.020903 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":40,"thread_start_us":356,"threads_started":1}
I20260806 19:34:37.021454 12102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.013042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:42401 (local address 127.10.25.2:47097)
0806 19:34:37.013962 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:37.013989 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:37.014174 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:37.014911 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:37.014915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:37.014945 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:37.015460 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.015474 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.016708 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.016714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.018824 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.018840 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.020317 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.020325 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.020505 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.020532 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 19:34:37.020676 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 19:34:37.020903 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":151,"thread_start_us":376,"threads_started":1}
I20260806 19:34:37.022687 11730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbba09874fbd4945a277479a7c4265a7" candidate_uuid: "7a1647c26a87478f915d648281ce4957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ad362f57c5b499daf39a74a3eb892c1" is_pre_election: true
I20260806 19:34:37.022679 12025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbba09874fbd4945a277479a7c4265a7" candidate_uuid: "7a1647c26a87478f915d648281ce4957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c125e721bd814a608f9ae49d2667edf3" is_pre_election: true
I20260806 19:34:37.023298 12025 raft_consensus.cc:2454] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a1647c26a87478f915d648281ce4957 in term 0.
I20260806 19:34:37.023303 11730 raft_consensus.cc:2454] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a1647c26a87478f915d648281ce4957 in term 0.
I20260806 19:34:37.024068 11802 leader_election.cc:302] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ad362f57c5b499daf39a74a3eb892c1, 7a1647c26a87478f915d648281ce4957; no voters:
I20260806 19:34:37.024598 12101 raft_consensus.cc:2781] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:34:37.024781 12101 raft_consensus.cc:482] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:34:37.024927 12101 raft_consensus.cc:3037] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:37.027374 12101 raft_consensus.cc:504] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.028055 12101 leader_election.cc:288] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [CANDIDATE]: Term 1 election: Requested vote from peers 1ad362f57c5b499daf39a74a3eb892c1 (127.10.25.1:42401), c125e721bd814a608f9ae49d2667edf3 (127.10.25.3:40035)
I20260806 19:34:37.028791 11730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbba09874fbd4945a277479a7c4265a7" candidate_uuid: "7a1647c26a87478f915d648281ce4957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ad362f57c5b499daf39a74a3eb892c1"
I20260806 19:34:37.028887 12025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbba09874fbd4945a277479a7c4265a7" candidate_uuid: "7a1647c26a87478f915d648281ce4957" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c125e721bd814a608f9ae49d2667edf3"
I20260806 19:34:37.029073 11730 raft_consensus.cc:3037] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:37.029165 12025 raft_consensus.cc:3037] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:37.032091 11730 raft_consensus.cc:2454] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a1647c26a87478f915d648281ce4957 in term 1.
I20260806 19:34:37.032714 11802 leader_election.cc:302] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ad362f57c5b499daf39a74a3eb892c1, 7a1647c26a87478f915d648281ce4957; no voters:
I20260806 19:34:37.032719 12025 raft_consensus.cc:2454] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a1647c26a87478f915d648281ce4957 in term 1.
I20260806 19:34:37.033106 12101 raft_consensus.cc:2781] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:34:37.033881 12101 raft_consensus.cc:686] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [term 1 LEADER]: Becoming Leader. State: Replica: 7a1647c26a87478f915d648281ce4957, State: Running, Role: LEADER
I20260806 19:34:37.035017 12101 consensus_queue.cc:237] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.042562 11560 catalog_manager.cc:5495] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a1647c26a87478f915d648281ce4957 (127.10.25.2). New cstate: current_term: 1 leader_uuid: "7a1647c26a87478f915d648281ce4957" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } health_report { overall_health: UNKNOWN } } }
I20260806 19:34:37.114657 12090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.107530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53624 (local address 127.10.25.2:46771)
0806 19:34:37.107806 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:37.107814 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:37.107968 (+ 154us) server_negotiation.cc:407] Connection header received
0806 19:34:37.108450 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:37.108455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:37.108559 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:37.108797 (+ 238us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:34:37.109243 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.110138 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.111669 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.112157 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.112335 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.112976 (+ 641us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:34:37.113868 (+ 892us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:34:37.114053 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:37.114230 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 19:34:37.114333 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260806 19:34:37.126266 12025 raft_consensus.cc:1261] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3 [term 1 FOLLOWER]: Refusing update from remote peer 7a1647c26a87478f915d648281ce4957: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:34:37.126288 11730 raft_consensus.cc:1261] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1 [term 1 FOLLOWER]: Refusing update from remote peer 7a1647c26a87478f915d648281ce4957: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:34:37.127519 12104 consensus_queue.cc:1035] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:34:37.128082 12101 consensus_queue.cc:1035] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:34:37.146793 12107 mvcc.cc:205] Tried to move back new op lower bound from 7315639816696524800 to 7315639816341368832. Current Snapshot: MvccSnapshot[applied={T|T < 7315639816696524800}]
I20260806 19:34:37.146796 12109 mvcc.cc:205] Tried to move back new op lower bound from 7315639816696524800 to 7315639816341368832. Current Snapshot: MvccSnapshot[applied={T|T < 7315639816696524800}]
I20260806 19:34:37.172487 11559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:34:37.172891 11559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:34:37.178263 11559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa: Sending DeleteTablet for 3 replicas of tablet cbba09874fbd4945a277479a7c4265a7
I20260806 19:34:37.179945 11851 tablet_service.cc:1514] Processing DeleteTablet for tablet cbba09874fbd4945a277479a7c4265a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:37 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:53618
I20260806 19:34:37.179950 11705 tablet_service.cc:1514] Processing DeleteTablet for tablet cbba09874fbd4945a277479a7c4265a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:37 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:59450
I20260806 19:34:37.181288 12116 tablet_replica.cc:330] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957: stopping tablet replica
I20260806 19:34:37.181654 11999 tablet_service.cc:1514] Processing DeleteTablet for tablet cbba09874fbd4945a277479a7c4265a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:37 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:42930
I20260806 19:34:37.182142 12116 raft_consensus.cc:2229] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:34:37.183261 12116 raft_consensus.cc:2258] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:37.187119 12116 ts_tablet_manager.cc:1905] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:37.187233 11559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:34:37.188155 11559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:34:37.196861 12117 tablet_replica.cc:330] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1: stopping tablet replica
I20260806 19:34:37.197472 12117 raft_consensus.cc:2229] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:34:37.198143 12117 raft_consensus.cc:2258] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:37.198588 12116 ts_tablet_manager.cc:1918] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:34:37.198966 12116 log.cc:1196] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/cbba09874fbd4945a277479a7c4265a7
I20260806 19:34:37.199693 12116 ts_tablet_manager.cc:1939] T cbba09874fbd4945a277479a7c4265a7 P 7a1647c26a87478f915d648281ce4957: Deleting consensus metadata
I20260806 19:34:37.201093 11705 tablet_service.cc:1467] Processing CreateTablet for tablet 29f5325b96b74b6fb0d1770fb7c81fe3 (DEFAULT_TABLE table=test-table [id=8eecd1aa674f45ab953c0300f3c1ef8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:37.201447 11851 tablet_service.cc:1467] Processing CreateTablet for tablet 29f5325b96b74b6fb0d1770fb7c81fe3 (DEFAULT_TABLE table=test-table [id=8eecd1aa674f45ab953c0300f3c1ef8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:37.201910 11544 catalog_manager.cc:4841] TS 7a1647c26a87478f915d648281ce4957 (127.10.25.2:46771): tablet cbba09874fbd4945a277479a7c4265a7 (table test-table [id=fe0c6b7cd0404e03bdb986f675af6a65]) successfully deleted
I20260806 19:34:37.202047 11851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29f5325b96b74b6fb0d1770fb7c81fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:37.202411 11705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29f5325b96b74b6fb0d1770fb7c81fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:37.203595 11998 tablet_service.cc:1467] Processing CreateTablet for tablet 29f5325b96b74b6fb0d1770fb7c81fe3 (DEFAULT_TABLE table=test-table [id=8eecd1aa674f45ab953c0300f3c1ef8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:37.204231 11998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29f5325b96b74b6fb0d1770fb7c81fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:37.204690 12118 tablet_replica.cc:330] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3: stopping tablet replica
I20260806 19:34:37.205184 12118 raft_consensus.cc:2229] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:34:37.205868 12118 raft_consensus.cc:2258] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:37.212484 12117 ts_tablet_manager.cc:1905] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:37.212661 12095 tablet_bootstrap.cc:492] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3: Bootstrap starting.
I20260806 19:34:37.214681 12095 tablet_bootstrap.cc:654] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:37.215457 12093 tablet_bootstrap.cc:492] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957: Bootstrap starting.
I20260806 19:34:37.216908 12094 tablet_bootstrap.cc:492] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1: Bootstrap starting.
I20260806 19:34:37.217662 12093 tablet_bootstrap.cc:654] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:37.219146 12094 tablet_bootstrap.cc:654] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:37.226749 12095 tablet_bootstrap.cc:492] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3: No bootstrap required, opened a new log
I20260806 19:34:37.226986 12095 ts_tablet_manager.cc:1397] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.011s
I20260806 19:34:37.228019 12095 raft_consensus.cc:348] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.228320 12095 raft_consensus.cc:374] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:37.228435 12095 raft_consensus.cc:729] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c125e721bd814a608f9ae49d2667edf3, State: Initialized, Role: FOLLOWER
I20260806 19:34:37.228842 12095 consensus_queue.cc:260] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.230549 12095 ts_tablet_manager.cc:1428] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 19:34:37.233016 12118 ts_tablet_manager.cc:1905] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:37.233968 12093 tablet_bootstrap.cc:492] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957: No bootstrap required, opened a new log
I20260806 19:34:37.234229 12093 ts_tablet_manager.cc:1397] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.011s
I20260806 19:34:37.235533 12093 raft_consensus.cc:348] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.235809 12093 raft_consensus.cc:374] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:37.235937 12093 raft_consensus.cc:729] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a1647c26a87478f915d648281ce4957, State: Initialized, Role: FOLLOWER
I20260806 19:34:37.237087 12093 consensus_queue.cc:260] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.238039 12093 ts_tablet_manager.cc:1428] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260806 19:34:37.239831 12094 tablet_bootstrap.cc:492] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1: No bootstrap required, opened a new log
I20260806 19:34:37.240149 12094 ts_tablet_manager.cc:1397] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.006s
I20260806 19:34:37.240840 12117 ts_tablet_manager.cc:1918] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:34:37.241098 12117 log.cc:1196] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/cbba09874fbd4945a277479a7c4265a7
I20260806 19:34:37.241617 12094 raft_consensus.cc:348] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.241882 12094 raft_consensus.cc:374] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:37.242015 12094 raft_consensus.cc:729] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ad362f57c5b499daf39a74a3eb892c1, State: Initialized, Role: FOLLOWER
I20260806 19:34:37.242406 12094 consensus_queue.cc:260] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.243088 12094 ts_tablet_manager.cc:1428] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 19:34:37.246112 12117 ts_tablet_manager.cc:1939] T cbba09874fbd4945a277479a7c4265a7 P 1ad362f57c5b499daf39a74a3eb892c1: Deleting consensus metadata
I20260806 19:34:37.246093 12118 ts_tablet_manager.cc:1918] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:34:37.246310 12118 log.cc:1196] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/cbba09874fbd4945a277479a7c4265a7
I20260806 19:34:37.246987 12118 ts_tablet_manager.cc:1939] T cbba09874fbd4945a277479a7c4265a7 P c125e721bd814a608f9ae49d2667edf3: Deleting consensus metadata
I20260806 19:34:37.248807 11546 catalog_manager.cc:4841] TS 1ad362f57c5b499daf39a74a3eb892c1 (127.10.25.1:42401): tablet cbba09874fbd4945a277479a7c4265a7 (table test-table [id=fe0c6b7cd0404e03bdb986f675af6a65]) successfully deleted
I20260806 19:34:37.249184 11546 catalog_manager.cc:4841] TS c125e721bd814a608f9ae49d2667edf3 (127.10.25.3:40035): tablet cbba09874fbd4945a277479a7c4265a7 (table test-table [id=fe0c6b7cd0404e03bdb986f675af6a65]) successfully deleted
I20260806 19:34:37.259043 12100 raft_consensus.cc:482] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:34:37.259406 12100 raft_consensus.cc:504] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.261842 12100 leader_election.cc:288] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a1647c26a87478f915d648281ce4957 (127.10.25.2:46771), c125e721bd814a608f9ae49d2667edf3 (127.10.25.3:40035)
I20260806 19:34:37.283010 12090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.262960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:56775 (local address 127.10.25.2:46771)
0806 19:34:37.263155 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:37.263161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:37.274221 (+ 11060us) server_negotiation.cc:407] Connection header received
0806 19:34:37.274581 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:37.274590 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:37.274677 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:37.274769 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:37.275731 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.277020 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.280091 (+ 3071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.281536 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.281671 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.281782 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:37.282478 (+ 696us) server_negotiation.cc:299] Negotiation successful
0806 19:34:37.282656 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 19:34:37.283867 12091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.262922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:47121 (local address 127.10.25.3:40035)
0806 19:34:37.263176 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:37.263185 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:37.274827 (+ 11642us) server_negotiation.cc:407] Connection header received
0806 19:34:37.275187 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:37.275194 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:37.275294 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:37.275365 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:37.276436 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.277755 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.280673 (+ 2918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.282292 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.282404 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.282503 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:37.283407 (+ 904us) server_negotiation.cc:299] Negotiation successful
0806 19:34:37.283571 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 19:34:37.283900 12124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.262258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:46771 (local address 127.10.25.1:56775)
0806 19:34:37.274051 (+ 11793us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:37.274082 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:37.274363 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:37.275047 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:37.275052 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:37.275090 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:37.275508 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.275521 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.277281 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.277287 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.279847 (+ 2560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.279869 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.282027 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.282034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.282256 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.282300 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 19:34:37.283331 (+ 1031us) client_negotiation.cc:240] Negotiation successful
0806 19:34:37.283506 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11609,"thread_start_us":11434,"threads_started":1}
I20260806 19:34:37.285342 12123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.262349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:40035 (local address 127.10.25.1:47121)
0806 19:34:37.274661 (+ 12312us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:37.274684 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:37.274975 (+ 291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:37.275637 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:37.275642 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:37.275670 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:37.276198 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.276222 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.277979 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.277984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.280453 (+ 2469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.280471 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.282758 (+ 2287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.282766 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.282973 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.283008 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 19:34:37.284713 (+ 1705us) client_negotiation.cc:240] Negotiation successful
0806 19:34:37.284890 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12170,"mutex_wait_us":1,"thread_start_us":11972,"threads_started":1}
I20260806 19:34:37.285051 11873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29f5325b96b74b6fb0d1770fb7c81fe3" candidate_uuid: "1ad362f57c5b499daf39a74a3eb892c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a1647c26a87478f915d648281ce4957" is_pre_election: true
I20260806 19:34:37.285651 11873 raft_consensus.cc:2454] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ad362f57c5b499daf39a74a3eb892c1 in term 0.
I20260806 19:34:37.286273 12025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29f5325b96b74b6fb0d1770fb7c81fe3" candidate_uuid: "1ad362f57c5b499daf39a74a3eb892c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c125e721bd814a608f9ae49d2667edf3" is_pre_election: true
I20260806 19:34:37.286537 12025 raft_consensus.cc:2454] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ad362f57c5b499daf39a74a3eb892c1 in term 0.
I20260806 19:34:37.286509 11653 leader_election.cc:302] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ad362f57c5b499daf39a74a3eb892c1, 7a1647c26a87478f915d648281ce4957; no voters:
I20260806 19:34:37.287055 12100 raft_consensus.cc:2781] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:34:37.287220 12100 raft_consensus.cc:482] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:34:37.287343 12100 raft_consensus.cc:3037] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:37.291541 12100 raft_consensus.cc:504] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.292524 12100 leader_election.cc:288] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [CANDIDATE]: Term 1 election: Requested vote from peers 7a1647c26a87478f915d648281ce4957 (127.10.25.2:46771), c125e721bd814a608f9ae49d2667edf3 (127.10.25.3:40035)
I20260806 19:34:37.293557 12025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29f5325b96b74b6fb0d1770fb7c81fe3" candidate_uuid: "1ad362f57c5b499daf39a74a3eb892c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c125e721bd814a608f9ae49d2667edf3"
I20260806 19:34:37.293591 11873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29f5325b96b74b6fb0d1770fb7c81fe3" candidate_uuid: "1ad362f57c5b499daf39a74a3eb892c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a1647c26a87478f915d648281ce4957"
I20260806 19:34:37.293833 12025 raft_consensus.cc:3037] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:37.293876 11873 raft_consensus.cc:3037] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:37.297394 12025 raft_consensus.cc:2454] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ad362f57c5b499daf39a74a3eb892c1 in term 1.
I20260806 19:34:37.297582 11873 raft_consensus.cc:2454] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ad362f57c5b499daf39a74a3eb892c1 in term 1.
I20260806 19:34:37.298043 11654 leader_election.cc:302] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ad362f57c5b499daf39a74a3eb892c1, c125e721bd814a608f9ae49d2667edf3; no voters:
I20260806 19:34:37.298452 12100 raft_consensus.cc:2781] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:34:37.299124 12100 raft_consensus.cc:686] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [term 1 LEADER]: Becoming Leader. State: Replica: 1ad362f57c5b499daf39a74a3eb892c1, State: Running, Role: LEADER
I20260806 19:34:37.299643 12100 consensus_queue.cc:237] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.304214 11559 catalog_manager.cc:5495] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ad362f57c5b499daf39a74a3eb892c1 (127.10.25.1). New cstate: current_term: 1 leader_uuid: "1ad362f57c5b499daf39a74a3eb892c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } health_report { overall_health: UNKNOWN } } }
Aug 06 19:34:37 dist-test-slave-ss4n krb5kdc[11510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044874, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:37.371536 12133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.344469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43935 (local address 127.0.0.1:33920)
0806 19:34:37.345587 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:37.345619 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:37.345773 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:37.346328 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:37.346332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:37.347155 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:37.347395 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.347405 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.349246 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.349251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.350389 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.350405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.351679 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.351686 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.351857 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.358421 (+ 6564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:37.358461 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:37.361361 (+ 2900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:37.367293 (+ 5932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:37.367313 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:37.367320 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:37.368349 (+ 1029us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:37.368970 (+ 621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:37.368974 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:37.368977 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:37.369194 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:37.370125 (+ 931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:37.370136 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:37.370520 (+ 384us) client_negotiation.cc:769] Sending connection context
0806 19:34:37.370763 (+ 243us) client_negotiation.cc:240] Negotiation successful
0806 19:34:37.371133 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":936,"thread_start_us":758,"threads_started":1}
I20260806 19:34:37.372102 12132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.344458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33920 (local address 127.10.25.62:43935)
0806 19:34:37.345296 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:37.345305 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:37.345725 (+ 420us) server_negotiation.cc:407] Connection header received
0806 19:34:37.345905 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:37.345912 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:37.346005 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:37.346169 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:37.347610 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.349027 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.350589 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.351379 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.351556 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.361764 (+ 10208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:37.361773 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:37.361779 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:37.361832 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:37.367016 (+ 5184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:37.368572 (+ 1556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:37.368580 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:37.368585 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:37.368790 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:37.369346 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:37.369352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:37.369356 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:37.369828 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:37.370008 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:37.370959 (+ 951us) server_negotiation.cc:299] Negotiation successful
0806 19:34:37.371397 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":640,"thread_start_us":421,"threads_started":1}
I20260806 19:34:37.379140 11559 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:33920:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 19:34:37.380865 11559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:34:37.389858 11705 tablet_service.cc:1467] Processing CreateTablet for tablet 8d817b0c503841708904eda102cf3170 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6616a9c940e4f28a4ec456b9d2f86e2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:34:37.390345 11998 tablet_service.cc:1467] Processing CreateTablet for tablet 8d817b0c503841708904eda102cf3170 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6616a9c940e4f28a4ec456b9d2f86e2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:34:37.390605 11705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d817b0c503841708904eda102cf3170. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:37.390897 11998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d817b0c503841708904eda102cf3170. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:37.395278 11851 tablet_service.cc:1467] Processing CreateTablet for tablet 8d817b0c503841708904eda102cf3170 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6616a9c940e4f28a4ec456b9d2f86e2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:34:37.396111 11851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d817b0c503841708904eda102cf3170. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:37.399637 12095 tablet_bootstrap.cc:492] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3: Bootstrap starting.
I20260806 19:34:37.401294 12095 tablet_bootstrap.cc:654] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:37.402588 12094 tablet_bootstrap.cc:492] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1: Bootstrap starting.
I20260806 19:34:37.405613 12094 tablet_bootstrap.cc:654] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:37.411345 12093 tablet_bootstrap.cc:492] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957: Bootstrap starting.
I20260806 19:34:37.413847 12093 tablet_bootstrap.cc:654] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:37.414165 12095 tablet_bootstrap.cc:492] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3: No bootstrap required, opened a new log
I20260806 19:34:37.414414 12095 ts_tablet_manager.cc:1397] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.006s
I20260806 19:34:37.415709 12095 raft_consensus.cc:348] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.416083 12095 raft_consensus.cc:374] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:37.416186 12095 raft_consensus.cc:729] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c125e721bd814a608f9ae49d2667edf3, State: Initialized, Role: FOLLOWER
I20260806 19:34:37.416591 12095 consensus_queue.cc:260] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.417181 12099 tablet_replica.cc:439] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } } }
I20260806 19:34:37.417627 12099 tablet_replica.cc:442] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:34:37.417989 12094 tablet_bootstrap.cc:492] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1: No bootstrap required, opened a new log
I20260806 19:34:37.418211 12094 ts_tablet_manager.cc:1397] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.005s
I20260806 19:34:37.419137 12094 raft_consensus.cc:348] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.419466 12094 raft_consensus.cc:374] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:37.419570 12094 raft_consensus.cc:729] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ad362f57c5b499daf39a74a3eb892c1, State: Initialized, Role: FOLLOWER
I20260806 19:34:37.419883 12094 consensus_queue.cc:260] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.420755 12100 tablet_replica.cc:439] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } } }
I20260806 19:34:37.421205 12100 tablet_replica.cc:442] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:34:37.421319 12094 ts_tablet_manager.cc:1428] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260806 19:34:37.422492 12095 ts_tablet_manager.cc:1428] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260806 19:34:37.431598 12093 tablet_bootstrap.cc:492] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957: No bootstrap required, opened a new log
I20260806 19:34:37.431844 12093 ts_tablet_manager.cc:1397] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.011s
I20260806 19:34:37.432755 12093 raft_consensus.cc:348] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.433050 12093 raft_consensus.cc:374] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:37.433138 12093 raft_consensus.cc:729] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a1647c26a87478f915d648281ce4957, State: Initialized, Role: FOLLOWER
I20260806 19:34:37.433429 12093 consensus_queue.cc:260] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.433966 12099 raft_consensus.cc:482] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:34:37.433957 12104 tablet_replica.cc:439] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } } }
I20260806 19:34:37.434327 12104 tablet_replica.cc:442] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:34:37.434222 12099 raft_consensus.cc:504] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.435958 12093 ts_tablet_manager.cc:1428] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260806 19:34:37.436422 12099 leader_election.cc:288] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ad362f57c5b499daf39a74a3eb892c1 (127.10.25.1:42401), 7a1647c26a87478f915d648281ce4957 (127.10.25.2:46771)
I20260806 19:34:37.447976 12090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.437528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:45071 (local address 127.10.25.2:46771)
0806 19:34:37.438592 (+ 1064us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:37.438602 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:37.439124 (+ 522us) server_negotiation.cc:407] Connection header received
0806 19:34:37.439607 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:37.439612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:37.439827 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:37.439904 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:37.441117 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.442498 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.444998 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.446586 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.446889 (+ 303us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.447045 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:37.447490 (+ 445us) server_negotiation.cc:299] Negotiation successful
0806 19:34:37.447662 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":834}
I20260806 19:34:37.448077 12138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.436955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:46771 (local address 127.10.25.3:45071)
0806 19:34:37.438949 (+ 1994us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:37.438974 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:37.439267 (+ 293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:37.440209 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:37.440214 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:37.440254 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:37.440865 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.440886 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.442692 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.442700 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.444758 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.444774 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.446973 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.446981 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.447250 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.447292 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 19:34:37.447453 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 19:34:37.447630 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1706,"mutex_wait_us":31,"thread_start_us":4630,"threads_started":1}
I20260806 19:34:37.448931 11873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d817b0c503841708904eda102cf3170" candidate_uuid: "c125e721bd814a608f9ae49d2667edf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a1647c26a87478f915d648281ce4957" is_pre_election: true
I20260806 19:34:37.449198 11873 raft_consensus.cc:2454] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c125e721bd814a608f9ae49d2667edf3 in term 0.
I20260806 19:34:37.449774 11950 leader_election.cc:302] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a1647c26a87478f915d648281ce4957, c125e721bd814a608f9ae49d2667edf3; no voters:
I20260806 19:34:37.450217 12099 raft_consensus.cc:2781] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:34:37.450317 12099 raft_consensus.cc:482] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:34:37.450439 12099 raft_consensus.cc:3037] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:37.454013 12099 raft_consensus.cc:504] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.454886 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.438462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:41523 (local address 127.10.25.1:42401)
0806 19:34:37.438674 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:37.438682 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:37.442098 (+ 3416us) server_negotiation.cc:407] Connection header received
0806 19:34:37.442422 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:37.442429 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:37.442519 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:37.442592 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:37.443698 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.445027 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.448062 (+ 3035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.450011 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.450144 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.450289 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:37.454325 (+ 4036us) server_negotiation.cc:299] Negotiation successful
0806 19:34:37.454524 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 19:34:37.455010 12137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.437841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:42401 (local address 127.10.25.3:41523)
0806 19:34:37.441967 (+ 4126us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:37.441997 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:37.442241 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:37.442879 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:37.442885 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:37.442922 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:37.443456 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.443472 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.445238 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.445244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.447805 (+ 2561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.447825 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.453792 (+ 5967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.453800 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.454050 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.454094 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 19:34:37.454217 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 19:34:37.454512 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3949,"mutex_wait_us":58,"thread_start_us":834,"threads_started":1}
I20260806 19:34:37.455885 11730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d817b0c503841708904eda102cf3170" candidate_uuid: "c125e721bd814a608f9ae49d2667edf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ad362f57c5b499daf39a74a3eb892c1" is_pre_election: true
I20260806 19:34:37.456069 11729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d817b0c503841708904eda102cf3170" candidate_uuid: "c125e721bd814a608f9ae49d2667edf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ad362f57c5b499daf39a74a3eb892c1"
I20260806 19:34:37.456209 11730 raft_consensus.cc:2454] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c125e721bd814a608f9ae49d2667edf3 in term 0.
I20260806 19:34:37.456547 11873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d817b0c503841708904eda102cf3170" candidate_uuid: "c125e721bd814a608f9ae49d2667edf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a1647c26a87478f915d648281ce4957"
I20260806 19:34:37.456831 11873 raft_consensus.cc:3037] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:37.459713 11873 raft_consensus.cc:2454] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c125e721bd814a608f9ae49d2667edf3 in term 1.
I20260806 19:34:37.459997 12099 leader_election.cc:288] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [CANDIDATE]: Term 1 election: Requested vote from peers 1ad362f57c5b499daf39a74a3eb892c1 (127.10.25.1:42401), 7a1647c26a87478f915d648281ce4957 (127.10.25.2:46771)
I20260806 19:34:37.460362 11950 leader_election.cc:302] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7a1647c26a87478f915d648281ce4957, c125e721bd814a608f9ae49d2667edf3; no voters: 1ad362f57c5b499daf39a74a3eb892c1
I20260806 19:34:37.460803 12099 raft_consensus.cc:2781] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:34:37.461371 12099 raft_consensus.cc:686] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [term 1 LEADER]: Becoming Leader. State: Replica: c125e721bd814a608f9ae49d2667edf3, State: Running, Role: LEADER
I20260806 19:34:37.461848 12099 consensus_queue.cc:237] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } }
I20260806 19:34:37.465334 12139 tablet_replica.cc:439] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3: TxnStatusTablet state changed. Reason: New leader c125e721bd814a608f9ae49d2667edf3. Latest consensus state: current_term: 1 leader_uuid: "c125e721bd814a608f9ae49d2667edf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } } }
I20260806 19:34:37.465742 12139 tablet_replica.cc:442] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:34:37.466041 11559 catalog_manager.cc:5495] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 reported cstate change: term changed from 0 to 1, leader changed from <none> to c125e721bd814a608f9ae49d2667edf3 (127.10.25.3). New cstate: current_term: 1 leader_uuid: "c125e721bd814a608f9ae49d2667edf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } health_report { overall_health: HEALTHY } } }
I20260806 19:34:37.466645 12141 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 19:34:37.467010 12141 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 19:34:37.509737 12133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.500545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:40035 (local address 127.0.0.1:42934)
0806 19:34:37.500963 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:37.501015 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:37.501259 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:37.502077 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:37.502083 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:37.502098 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:34:37.502348 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.502361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.503960 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.503966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.505808 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.505828 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.507176 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.507183 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.507453 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.507594 (+ 141us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:34:37.508946 (+ 1352us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:34:37.508977 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 19:34:37.509086 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 19:34:37.509304 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 19:34:37.510636 12091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.500236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42934 (local address 127.10.25.3:40035)
0806 19:34:37.500511 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:37.500519 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:37.501149 (+ 630us) server_negotiation.cc:407] Connection header received
0806 19:34:37.501586 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:37.501592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:37.501711 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:37.501863 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:34:37.502536 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.503727 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.506046 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.506830 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.506994 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.507867 (+ 873us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:34:37.508721 (+ 854us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:34:37.508873 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:37.510112 (+ 1239us) server_negotiation.cc:299] Negotiation successful
0806 19:34:37.510214 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 19:34:37.516505 11729 raft_consensus.cc:3037] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:37.516970 11873 raft_consensus.cc:1261] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957 [term 1 FOLLOWER]: Refusing update from remote peer c125e721bd814a608f9ae49d2667edf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:34:37.518067 12099 consensus_queue.cc:1035] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:34:37.520758 11729 raft_consensus.cc:1261] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1 [term 1 FOLLOWER]: Refusing update from remote peer c125e721bd814a608f9ae49d2667edf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:34:37.521907 12099 consensus_queue.cc:1035] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:34:37.529922 12100 tablet_replica.cc:439] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1: TxnStatusTablet state changed. Reason: New leader c125e721bd814a608f9ae49d2667edf3. Latest consensus state: current_term: 1 leader_uuid: "c125e721bd814a608f9ae49d2667edf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } } }
I20260806 19:34:37.531577 12100 tablet_replica.cc:442] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:34:37.535174 12104 tablet_replica.cc:439] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957: TxnStatusTablet state changed. Reason: New leader c125e721bd814a608f9ae49d2667edf3. Latest consensus state: current_term: 1 leader_uuid: "c125e721bd814a608f9ae49d2667edf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } } }
I20260806 19:34:37.535529 12104 tablet_replica.cc:442] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:34:37.553768 12139 tablet_replica.cc:439] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c125e721bd814a608f9ae49d2667edf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } } }
I20260806 19:34:37.554260 12139 tablet_replica.cc:442] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:34:37.554786 12139 tablet_replica.cc:439] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c125e721bd814a608f9ae49d2667edf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } } }
I20260806 19:34:37.555157 12139 tablet_replica.cc:442] T 8d817b0c503841708904eda102cf3170 P c125e721bd814a608f9ae49d2667edf3: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:34:37.556038 12108 mvcc.cc:205] Tried to move back new op lower bound from 7315639818298695680 to 7315639818088726528. Current Snapshot: MvccSnapshot[applied={T|T < 7315639818298695680 or (T in {7315639818298695680})}]
I20260806 19:34:37.565073 12100 tablet_replica.cc:439] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c125e721bd814a608f9ae49d2667edf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } } }
I20260806 19:34:37.566457 12100 tablet_replica.cc:442] T 8d817b0c503841708904eda102cf3170 P 1ad362f57c5b499daf39a74a3eb892c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:34:37.571980 12104 tablet_replica.cc:439] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c125e721bd814a608f9ae49d2667edf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ad362f57c5b499daf39a74a3eb892c1" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42401 } } peers { permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 } } peers { permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 } } }
I20260806 19:34:37.572407 12104 tablet_replica.cc:442] T 8d817b0c503841708904eda102cf3170 P 7a1647c26a87478f915d648281ce4957: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:34:37.572718 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.556916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59464 (local address 127.10.25.1:42401)
0806 19:34:37.557280 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:37.557287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:37.557308 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:34:37.557424 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:37.557443 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:37.557790 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:37.558040 (+ 250us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:34:37.559329 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.561220 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.565051 (+ 3831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.567428 (+ 2377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.567687 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.568904 (+ 1217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:34:37.570444 (+ 1540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:34:37.570719 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:37.571750 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0806 19:34:37.571897 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260806 19:34:37.635232 12091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.612141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:35653 (local address 127.10.25.3:40035)
0806 19:34:37.612367 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:37.612374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:37.624345 (+ 11971us) server_negotiation.cc:407] Connection header received
0806 19:34:37.624894 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:37.624902 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:37.625005 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:37.625086 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:34:37.626366 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.627879 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.631037 (+ 3158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.632087 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.632285 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.634083 (+ 1798us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:34:37.634293 (+ 210us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:34:37.634386 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:37.634730 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 19:34:37.634836 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 19:34:37.636380 12150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.612544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:40035 (local address 127.10.25.1:35653)
0806 19:34:37.624170 (+ 11626us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:37.624198 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:37.624651 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:37.625361 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:37.625366 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:37.625396 (+ 30us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:34:37.626050 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.626067 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.628186 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.628192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.630736 (+ 2544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.630761 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.633302 (+ 2541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.633310 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.633584 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.633697 (+ 113us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:34:37.634531 (+ 834us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:34:37.634546 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 19:34:37.635665 (+ 1119us) client_negotiation.cc:240] Negotiation successful
0806 19:34:37.635871 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11238,"thread_start_us":10517,"threads_started":1}
I20260806 19:34:37.653852 11873 raft_consensus.cc:1261] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957 [term 1 FOLLOWER]: Refusing update from remote peer 1ad362f57c5b499daf39a74a3eb892c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:34:37.653939 12025 raft_consensus.cc:1261] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3 [term 1 FOLLOWER]: Refusing update from remote peer 1ad362f57c5b499daf39a74a3eb892c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:34:37.654811 12125 consensus_queue.cc:1035] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a1647c26a87478f915d648281ce4957" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:34:37.655731 12100 consensus_queue.cc:1035] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c125e721bd814a608f9ae49d2667edf3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 40035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 19:34:37.775631 12011 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.GetTabletLocations -> {remote=127.10.25.62:43935, user_credentials={real_user=slave}} blocked reactor thread for 46130us
I20260806 19:34:37.792452 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.777885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:36315 (local address 127.10.25.1:42401)
0806 19:34:37.778116 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:37.778124 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:37.784125 (+ 6001us) server_negotiation.cc:407] Connection header received
0806 19:34:37.784390 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:37.784396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:37.784503 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:37.784614 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:34:37.785541 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.786803 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.789547 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.790373 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.790541 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.791444 (+ 903us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:34:37.791622 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:34:37.791702 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:37.791977 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 19:34:37.792084 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 19:34:37.794021 12157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.778105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:42401 (local address 127.10.25.3:36315)
0806 19:34:37.783934 (+ 5829us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:37.783978 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:37.784211 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:37.784717 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:37.784724 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:37.784743 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:34:37.785240 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.785258 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.786984 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.787059 (+ 75us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.789266 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:37.789291 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.790659 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.790668 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:37.790917 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.791066 (+ 149us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:34:37.791803 (+ 737us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:34:37.791814 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 19:34:37.793188 (+ 1374us) client_negotiation.cc:240] Negotiation successful
0806 19:34:37.793392 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5540,"thread_start_us":5328,"threads_started":1}
I20260806 19:34:37.885344 11559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:34:37.885869 11559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:34:37.890177 11559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bdb335339b5543a6b59d50bec57a58aa: Sending DeleteTablet for 3 replicas of tablet 29f5325b96b74b6fb0d1770fb7c81fe3
I20260806 19:34:37.891280 11851 tablet_service.cc:1514] Processing DeleteTablet for tablet 29f5325b96b74b6fb0d1770fb7c81fe3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:37 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:53618
I20260806 19:34:37.892367 11705 tablet_service.cc:1514] Processing DeleteTablet for tablet 29f5325b96b74b6fb0d1770fb7c81fe3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:37 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:59450
I20260806 19:34:37.899137 11998 tablet_service.cc:1514] Processing DeleteTablet for tablet 29f5325b96b74b6fb0d1770fb7c81fe3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:37 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:42930
I20260806 19:34:37.938985 12163 tablet_replica.cc:330] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3: stopping tablet replica
I20260806 19:34:37.939321 12163 raft_consensus.cc:2229] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:34:37.942026 12162 tablet_replica.cc:330] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1: stopping tablet replica
I20260806 19:34:37.942327 12162 raft_consensus.cc:2229] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:34:37.943058 12162 raft_consensus.cc:2258] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:37.943809 12163 raft_consensus.cc:2258] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:37.945541 12161 tablet_replica.cc:330] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957: stopping tablet replica
I20260806 19:34:37.945864 12161 raft_consensus.cc:2229] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:34:37.946162 12161 raft_consensus.cc:2258] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:37.947544 12161 ts_tablet_manager.cc:1905] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:37.950790 12162 ts_tablet_manager.cc:1905] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:37.945479 12163 ts_tablet_manager.cc:1905] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 19:34:37 dist-test-slave-ss4n krb5kdc[11510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044876, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:34:37.979568 12161 ts_tablet_manager.cc:1918] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:34:37.979760 12161 log.cc:1196] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/29f5325b96b74b6fb0d1770fb7c81fe3
I20260806 19:34:37.980254 12161 ts_tablet_manager.cc:1939] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 7a1647c26a87478f915d648281ce4957: Deleting consensus metadata
I20260806 19:34:37.981616 11544 catalog_manager.cc:4841] TS 7a1647c26a87478f915d648281ce4957 (127.10.25.2:46771): tablet 29f5325b96b74b6fb0d1770fb7c81fe3 (table test-table [id=8eecd1aa674f45ab953c0300f3c1ef8e]) successfully deleted
I20260806 19:34:37.982435 12163 ts_tablet_manager.cc:1918] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:34:37.982609 12163 log.cc:1196] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/29f5325b96b74b6fb0d1770fb7c81fe3
I20260806 19:34:37.983155 12163 ts_tablet_manager.cc:1939] T 29f5325b96b74b6fb0d1770fb7c81fe3 P c125e721bd814a608f9ae49d2667edf3: Deleting consensus metadata
I20260806 19:34:37.984516 11546 catalog_manager.cc:4841] TS c125e721bd814a608f9ae49d2667edf3 (127.10.25.3:40035): tablet 29f5325b96b74b6fb0d1770fb7c81fe3 (table test-table [id=8eecd1aa674f45ab953c0300f3c1ef8e]) successfully deleted
I20260806 19:34:37.987679 12162 ts_tablet_manager.cc:1918] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:34:37.987951 12162 log.cc:1196] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/29f5325b96b74b6fb0d1770fb7c81fe3
I20260806 19:34:37.988762 12162 ts_tablet_manager.cc:1939] T 29f5325b96b74b6fb0d1770fb7c81fe3 P 1ad362f57c5b499daf39a74a3eb892c1: Deleting consensus metadata
I20260806 19:34:37.989610 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:37.922637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59470 (local address 127.10.25.1:42401)
0806 19:34:37.922885 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:37.922892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:37.934780 (+ 11888us) server_negotiation.cc:407] Connection header received
0806 19:34:37.935838 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:37.935850 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:37.935992 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:37.936126 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:37.955874 (+ 19748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.957348 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.959875 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:37.960655 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:37.960798 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:37.978584 (+ 17786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:37.978623 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:37.978647 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:37.978721 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:37.984270 (+ 5549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:37.985587 (+ 1317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:37.985599 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:37.985603 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:37.985816 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:37.986381 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:37.986388 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:37.986394 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:37.987135 (+ 741us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:37.987536 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:37.988410 (+ 874us) server_negotiation.cc:299] Negotiation successful
0806 19:34:37.988934 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260806 19:34:37.991657 11546 catalog_manager.cc:4841] TS 1ad362f57c5b499daf39a74a3eb892c1 (127.10.25.1:42401): tablet 29f5325b96b74b6fb0d1770fb7c81fe3 (table test-table [id=8eecd1aa674f45ab953c0300f3c1ef8e]) successfully deleted
W20260806 19:34:37.993113 11774 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59470
I20260806 19:34:38.052089 12167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:38.010404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33936 (local address 127.10.25.62:43935)
0806 19:34:38.029966 (+ 19562us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:38.029974 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:38.030001 (+ 27us) server_negotiation.cc:407] Connection header received
0806 19:34:38.030682 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:38.030690 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:38.030786 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:38.030890 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:38.033431 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:38.034994 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:38.037465 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:38.038210 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:38.038356 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:38.042976 (+ 4620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:38.042985 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:38.042991 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:38.043051 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:38.047834 (+ 4783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:38.049007 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:38.049014 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:38.049019 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:38.049204 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:38.049772 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:38.049780 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:38.049783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:38.050201 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:38.050449 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:38.051289 (+ 840us) server_negotiation.cc:299] Negotiation successful
0806 19:34:38.051588 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19383,"thread_start_us":19721,"threads_started":1}
W20260806 19:34:38.053710 11559 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33936
I20260806 19:34:38.057336 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 11635
I20260806 19:34:38.079717 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:38.059329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59476 (local address 127.10.25.1:42401)
0806 19:34:38.059551 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:38.059558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:38.064254 (+ 4696us) server_negotiation.cc:407] Connection header received
0806 19:34:38.064482 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:38.064488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:38.064579 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:38.064707 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:38.066326 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:38.067232 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:38.068648 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:38.069413 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:38.069623 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:38.072385 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:38.072391 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:38.072395 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:38.072441 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:38.076209 (+ 3768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:38.077168 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:38.077175 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:38.077186 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:38.077370 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:38.077885 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:38.077895 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:38.077901 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:38.078223 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:38.078530 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:38.078950 (+ 420us) server_negotiation.cc:299] Negotiation successful
0806 19:34:38.079248 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 19:34:38.080528 11774 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59476
W20260806 19:34:38.081210 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 11635: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:34:38.081444 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 11635
I20260806 19:34:38.109258 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 11783
Aug 06 19:34:38 dist-test-slave-ss4n krb5kdc[11510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044876, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:34:38.132998 12170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:38.110176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53632 (local address 127.10.25.2:46771)
0806 19:34:38.110804 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:38.110808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:38.110825 (+ 17us) server_negotiation.cc:407] Connection header received
0806 19:34:38.110882 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:38.110886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:38.110945 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:38.111047 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:38.112943 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:38.114144 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:38.116545 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:38.117067 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:38.117198 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:38.123665 (+ 6467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:38.123697 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:38.123727 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:38.123797 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:38.128887 (+ 5090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:38.130191 (+ 1304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:38.130204 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:38.130208 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:38.130479 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:38.131001 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:38.131007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:38.131010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:38.131550 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:38.131917 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:38.132226 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 19:34:38.132523 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":368,"threads_started":1}
W20260806 19:34:38.134335 11923 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53632
W20260806 19:34:38.135300 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 11783: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:34:38.135528 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 11783
I20260806 19:34:38.163934 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 11932
Aug 06 19:34:38 dist-test-slave-ss4n krb5kdc[11510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044876, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:34:38.182838 12171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:38.164981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42938 (local address 127.10.25.3:40035)
0806 19:34:38.165734 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:38.165738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:38.165754 (+ 16us) server_negotiation.cc:407] Connection header received
0806 19:34:38.165809 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:38.165812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:38.165872 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:38.166011 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:38.167358 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:38.168225 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:38.169776 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:38.170244 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:38.170404 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:38.175470 (+ 5066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:38.175486 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:38.175504 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:38.175554 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:38.179511 (+ 3957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:38.180357 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:38.180366 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:38.180368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:38.180535 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:38.180946 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:38.180953 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:38.180961 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:38.181368 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:38.181796 (+ 428us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:38.182183 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 19:34:38.182433 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":493,"threads_started":1}
W20260806 19:34:38.183586 12069 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42938
W20260806 19:34:38.184407 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 11932: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:34:38.184618 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 11932
I20260806 19:34:38.210047 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 11525
I20260806 19:34:38.211341 11603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:34:38.340206 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 11525
2026-08-06T19:34:38Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5972 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:34:38 dist-test-slave-ss4n krb5kdc[12175](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:34:38 dist-test-slave-ss4n krb5kdc[12175](info): set up 2 sockets
Aug 06 19:34:38 dist-test-slave-ss4n krb5kdc[12175](info): commencing operation
krb5kdc: starting...
W20260806 19:34:39.597918 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.003s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:34:39 dist-test-slave-ss4n krb5kdc[12175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:34:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:34:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:34:39.850800 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:41915
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:37185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:41915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:40.224280 12191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:40.224697 12191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:40.224874 12191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:40.232604 12191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:34:40.232810 12191 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 19:34:40.232888 12191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:40.233017 12191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:40.233112 12191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:34:40.233217 12191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:34:40.243769 12191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:37185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:41915
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:41915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:40.245288 12191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:40.246759 12191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:40.260818 12196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:40.260916 12197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:40.260903 12200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:40.262511 12191 server_base.cc:1034] running on GCE node
I20260806 19:34:40.263423 12191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:40.265044 12191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:40.266515 12191 hybrid_clock.cc:648] HybridClock initialized: now 1786044880266391 us; error 75 us; skew 500 ppm
I20260806 19:34:40.267060 12191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:40 dist-test-slave-ss4n krb5kdc[12175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044880, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:40.271332 12191 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:34:40.273285 12191 webserver.cc:460] Webserver started at http://127.10.25.62:37889/ using document root <none> and password file <none>
I20260806 19:34:40.274690 12191 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:40.274852 12191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:40.275352 12191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:40.279755 12191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "6d1d41afcaaf4ee2bcd77d3913ba3a5d"
format_stamp: "Formatted at 2026-08-06 19:34:40 on dist-test-slave-ss4n"
server_key: "7e84d7828cb871120f38196ffff4030d"
server_key_iv: "547dcb683bf62c9272ba84175e12feb5"
server_key_version: "encryptionkey@0"
I20260806 19:34:40.280804 12191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "6d1d41afcaaf4ee2bcd77d3913ba3a5d"
format_stamp: "Formatted at 2026-08-06 19:34:40 on dist-test-slave-ss4n"
server_key: "7e84d7828cb871120f38196ffff4030d"
server_key_iv: "547dcb683bf62c9272ba84175e12feb5"
server_key_version: "encryptionkey@0"
I20260806 19:34:40.287401 12191 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260806 19:34:40.291852 12207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:40.294701 12191 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 19:34:40.295040 12191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "6d1d41afcaaf4ee2bcd77d3913ba3a5d"
format_stamp: "Formatted at 2026-08-06 19:34:40 on dist-test-slave-ss4n"
server_key: "7e84d7828cb871120f38196ffff4030d"
server_key_iv: "547dcb683bf62c9272ba84175e12feb5"
server_key_version: "encryptionkey@0"
I20260806 19:34:40.295373 12191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:40.321468 12191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:40.327445 12191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:40.327849 12191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:40.358044 12191 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:41915
I20260806 19:34:40.358119 12271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:41915 every 8 connection(s)
I20260806 19:34:40.360026 12191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:34:40.365414 12272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:40.369298 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 12191
I20260806 19:34:40.369647 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:34:40.370496 10340 external_mini_cluster.cc:1428] Setting key 54aefda8a6925b3825123345d5de2927
I20260806 19:34:40.383431 12272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d: Bootstrap starting.
I20260806 19:34:40.387499 12272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:40.389245 12272 log.cc:824] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d: Log is configured to *not* fsync() on all Append() calls
Aug 06 19:34:40 dist-test-slave-ss4n krb5kdc[12175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:40.392956 12272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d: No bootstrap required, opened a new log
I20260806 19:34:40.404335 12275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:40.373427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41068 (local address 127.10.25.62:41915)
0806 19:34:40.374688 (+ 1261us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:40.374714 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:40.374787 (+ 73us) server_negotiation.cc:407] Connection header received
0806 19:34:40.378113 (+ 3326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:40.378158 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:40.378855 (+ 697us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:40.379443 (+ 588us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:40.381583 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:40.383702 (+ 2119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:40.385386 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:40.386557 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:40.386798 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:40.392218 (+ 5420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:40.392227 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:40.392244 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:40.392291 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:40.396739 (+ 4448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:40.398384 (+ 1645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:40.398396 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:40.398410 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:40.398721 (+ 311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:40.399357 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:40.399362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:40.399367 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:40.400934 (+ 1567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:40.401464 (+ 530us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:40.402444 (+ 980us) server_negotiation.cc:299] Negotiation successful
0806 19:34:40.403101 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":881,"thread_start_us":463,"threads_started":1}
I20260806 19:34:40.404460 12272 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1d41afcaaf4ee2bcd77d3913ba3a5d" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41915 } }
I20260806 19:34:40.405002 12272 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:40.405148 12272 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d1d41afcaaf4ee2bcd77d3913ba3a5d, State: Initialized, Role: FOLLOWER
I20260806 19:34:40.405915 12272 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1d41afcaaf4ee2bcd77d3913ba3a5d" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41915 } }
I20260806 19:34:40.406219 12272 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:34:40.406378 12272 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:34:40.406584 12272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:40.409406 12272 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1d41afcaaf4ee2bcd77d3913ba3a5d" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41915 } }
I20260806 19:34:40.410099 12272 leader_election.cc:302] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d1d41afcaaf4ee2bcd77d3913ba3a5d; no voters:
I20260806 19:34:40.410822 12272 leader_election.cc:288] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:34:40.411011 12277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:34:40.411865 12277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [term 1 LEADER]: Becoming Leader. State: Replica: 6d1d41afcaaf4ee2bcd77d3913ba3a5d, State: Running, Role: LEADER
I20260806 19:34:40.412480 12277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1d41afcaaf4ee2bcd77d3913ba3a5d" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41915 } }
I20260806 19:34:40.413169 12272 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:34:40.416390 12278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d1d41afcaaf4ee2bcd77d3913ba3a5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1d41afcaaf4ee2bcd77d3913ba3a5d" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41915 } } }
I20260806 19:34:40.416512 12279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d1d41afcaaf4ee2bcd77d3913ba3a5d. Latest consensus state: current_term: 1 leader_uuid: "6d1d41afcaaf4ee2bcd77d3913ba3a5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1d41afcaaf4ee2bcd77d3913ba3a5d" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41915 } } }
I20260806 19:34:40.417454 12278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [sys.catalog]: This master's current role is: LEADER
I20260806 19:34:40.418004 12279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d [sys.catalog]: This master's current role is: LEADER
I20260806 19:34:40.423576 12287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:34:40.429821 12287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:34:40.442217 12287 catalog_manager.cc:1347] Generated new cluster ID: df76595b817f4fb696f41a385bf4a79d
I20260806 19:34:40.442427 12287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:34:40.461050 12287 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:34:40.462924 12287 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:34:40.475129 12287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d: Generated new TSK 0
I20260806 19:34:40.476210 12287 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:34:40.550290 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:41915
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:37185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:40.997351 12299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:40.997834 12299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:40.998027 12299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:41.007397 12299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:41.007612 12299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:41.007779 12299 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 19:34:41.007872 12299 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:34:41.008029 12299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:34:41.019804 12299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:37185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.25.62:41915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:41.021283 12299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:41.022764 12299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:41.038990 12305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:41.038938 12304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:41.038960 12307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:41.040396 12299 server_base.cc:1034] running on GCE node
I20260806 19:34:41.041131 12299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:41.042384 12299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:41.043672 12299 hybrid_clock.cc:648] HybridClock initialized: now 1786044881043584 us; error 78 us; skew 500 ppm
I20260806 19:34:41.044056 12299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:41 dist-test-slave-ss4n krb5kdc[12175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044881, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:41.047845 12299 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:34:41.049889 12299 webserver.cc:460] Webserver started at http://127.10.25.1:45315/ using document root <none> and password file <none>
I20260806 19:34:41.050936 12299 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:41.051164 12299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:41.051573 12299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:41.055035 12299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "3f0da279763745aa9289e16bc43ef630"
format_stamp: "Formatted at 2026-08-06 19:34:41 on dist-test-slave-ss4n"
server_key: "9f8123129baa3e58cb28883ebca5ef73"
server_key_iv: "6c64504a054486c949ce29fc46d95b82"
server_key_version: "encryptionkey@0"
I20260806 19:34:41.055943 12299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "3f0da279763745aa9289e16bc43ef630"
format_stamp: "Formatted at 2026-08-06 19:34:41 on dist-test-slave-ss4n"
server_key: "9f8123129baa3e58cb28883ebca5ef73"
server_key_iv: "6c64504a054486c949ce29fc46d95b82"
server_key_version: "encryptionkey@0"
I20260806 19:34:41.063133 12299 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 19:34:41.067708 12314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:41.069648 12299 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 19:34:41.069942 12299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "3f0da279763745aa9289e16bc43ef630"
format_stamp: "Formatted at 2026-08-06 19:34:41 on dist-test-slave-ss4n"
server_key: "9f8123129baa3e58cb28883ebca5ef73"
server_key_iv: "6c64504a054486c949ce29fc46d95b82"
server_key_version: "encryptionkey@0"
I20260806 19:34:41.070183 12299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:41.092470 12299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:41.097954 12299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:41.098313 12299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:41.102023 12299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:41.102205 12299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:41.102413 12299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:41.102571 12299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:34:41 dist-test-slave-ss4n krb5kdc[12175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044881, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:41.130748 12322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:41.102426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:41589 (local address 127.10.25.62:41915)
0806 19:34:41.102985 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:41.102992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:41.104425 (+ 1433us) server_negotiation.cc:407] Connection header received
0806 19:34:41.107008 (+ 2583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:41.107019 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:41.107140 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:41.107318 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:41.110202 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:41.111298 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:41.114413 (+ 3115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:41.114869 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:41.115039 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:41.120659 (+ 5620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:41.120665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:41.120668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:41.120708 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:41.124400 (+ 3692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:41.125644 (+ 1244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:41.125655 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:41.125663 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:41.125931 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:41.126851 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:41.126864 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:41.126870 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:41.127570 (+ 700us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:41.128004 (+ 434us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:41.129908 (+ 1904us) server_negotiation.cc:299] Negotiation successful
0806 19:34:41.130284 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":280,"threads_started":1}
I20260806 19:34:41.132040 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:41.103196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:41915 (local address 127.10.25.1:41589)
0806 19:34:41.104214 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:41.104264 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:41.106521 (+ 2257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:41.107704 (+ 1183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:41.107716 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:41.108702 (+ 986us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:41.109894 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:41.109937 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:41.111491 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:41.111507 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:41.114219 (+ 2712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:41.114239 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:41.115155 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:41.115165 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:41.115428 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:41.116415 (+ 987us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:41.116443 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:41.120307 (+ 3864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:41.124615 (+ 4308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:41.124634 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:41.124666 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:41.125433 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:41.126110 (+ 677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:41.126117 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:41.126123 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:41.126413 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:41.128121 (+ 1708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:41.128136 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:41.128826 (+ 690us) client_negotiation.cc:769] Sending connection context
0806 19:34:41.129668 (+ 842us) client_negotiation.cc:240] Negotiation successful
0806 19:34:41.130530 (+ 862us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":431,"threads_started":1}
I20260806 19:34:41.160681 12322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:41.140598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:52561 (local address 127.10.25.62:41915)
0806 19:34:41.140900 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:41.140910 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:41.142329 (+ 1419us) server_negotiation.cc:407] Connection header received
0806 19:34:41.142686 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:41.142695 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:41.142809 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:41.143005 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:41.145699 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:41.146467 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:41.147993 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:41.148456 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:41.148609 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:41.152079 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:41.152092 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:41.152100 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:41.152185 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:41.156933 (+ 4748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:41.158200 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:41.158205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:41.158208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:41.158425 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:41.158913 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:41.158918 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:41.158922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:41.159403 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:41.159599 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:41.160032 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 19:34:41.160284 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260806 19:34:41.160768 12380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:41.140534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:41915 (local address 127.10.25.1:52561)
0806 19:34:41.142117 (+ 1583us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:41.142147 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:41.142419 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:41.143079 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:41.143087 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:41.144748 (+ 1661us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:41.145401 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:41.145418 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:41.146666 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:41.146672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:41.147812 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:41.147827 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:41.148744 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:41.148752 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:41.148922 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:41.150060 (+ 1138us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:41.150082 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:41.151648 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:41.157186 (+ 5538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:41.157202 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:41.157206 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:41.158025 (+ 819us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:41.158593 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:41.158601 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:41.158607 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:41.158775 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:41.159539 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:41.159544 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:41.159758 (+ 214us) client_negotiation.cc:769] Sending connection context
0806 19:34:41.159907 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 19:34:41.160299 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1325,"thread_start_us":1078,"threads_started":1}
I20260806 19:34:41.174602 12299 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:43133
I20260806 19:34:41.174665 12439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:43133 every 8 connection(s)
I20260806 19:34:41.176417 12299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:34:41.179591 12440 heartbeater.cc:347] Connected to a master server at 127.10.25.62:41915
I20260806 19:34:41.180194 12440 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:41.181054 12440 heartbeater.cc:511] Master 127.10.25.62:41915 requested a full tablet report, sending...
I20260806 19:34:41.182910 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 12299
I20260806 19:34:41.183159 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:34:41.183990 10340 external_mini_cluster.cc:1428] Setting key b5ab0938b1801472e102a214968fc559
I20260806 19:34:41.184031 12226 ts_manager.cc:194] Registered new tserver with Master: 3f0da279763745aa9289e16bc43ef630 (127.10.25.1:43133)
I20260806 19:34:41.186172 12226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:41589
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:34:41.243534 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:41915
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:37185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:41.640190 12445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:41.640534 12445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:41.640661 12445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:41.649346 12445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:41.649545 12445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:41.649677 12445 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 19:34:41.649737 12445 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:34:41.649813 12445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:34:41.662745 12445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:37185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.25.62:41915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:41.664343 12445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:41.665993 12445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:41.683228 12451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:41.683210 12453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:41.684126 12450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:41.684465 12445 server_base.cc:1034] running on GCE node
I20260806 19:34:41.685249 12445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:41.686581 12445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:41.688066 12445 hybrid_clock.cc:648] HybridClock initialized: now 1786044881688001 us; error 54 us; skew 500 ppm
I20260806 19:34:41.688519 12445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:41 dist-test-slave-ss4n krb5kdc[12175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044881, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:41.692668 12445 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:34:41.694713 12445 webserver.cc:460] Webserver started at http://127.10.25.2:40429/ using document root <none> and password file <none>
I20260806 19:34:41.695541 12445 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:41.695724 12445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:41.696097 12445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:41.700049 12445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "575303e10edd446db43b7c363ecd132a"
format_stamp: "Formatted at 2026-08-06 19:34:41 on dist-test-slave-ss4n"
server_key: "c9f15310d39f0d4f9b19a6f2bdfd3c99"
server_key_iv: "7c24c1f2d0ea02cfa3456ea88ac994b3"
server_key_version: "encryptionkey@0"
I20260806 19:34:41.701256 12445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "575303e10edd446db43b7c363ecd132a"
format_stamp: "Formatted at 2026-08-06 19:34:41 on dist-test-slave-ss4n"
server_key: "c9f15310d39f0d4f9b19a6f2bdfd3c99"
server_key_iv: "7c24c1f2d0ea02cfa3456ea88ac994b3"
server_key_version: "encryptionkey@0"
I20260806 19:34:41.708559 12445 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 19:34:41.713244 12460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:41.716114 12445 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 19:34:41.716401 12445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "575303e10edd446db43b7c363ecd132a"
format_stamp: "Formatted at 2026-08-06 19:34:41 on dist-test-slave-ss4n"
server_key: "c9f15310d39f0d4f9b19a6f2bdfd3c99"
server_key_iv: "7c24c1f2d0ea02cfa3456ea88ac994b3"
server_key_version: "encryptionkey@0"
I20260806 19:34:41.716683 12445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:41.734656 12445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:41.740733 12445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:41.741217 12445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:41.744730 12445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:41.744912 12445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:41.745065 12445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:41.745179 12445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:34:41 dist-test-slave-ss4n krb5kdc[12175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044881, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:41.774334 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:41.745577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:47169 (local address 127.10.25.62:41915)
0806 19:34:41.746260 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:41.746267 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:41.747576 (+ 1309us) server_negotiation.cc:407] Connection header received
0806 19:34:41.749908 (+ 2332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:41.749914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:41.750031 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:41.750172 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:41.752864 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:41.754001 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:41.756999 (+ 2998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:41.757752 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:41.757903 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:41.762379 (+ 4476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:41.762391 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:41.762399 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:41.762454 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:41.767864 (+ 5410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:41.770025 (+ 2161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:41.770031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:41.770034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:41.770245 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:41.771190 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:41.771195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:41.771198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:41.771615 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:41.771901 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:41.773406 (+ 1505us) server_negotiation.cc:299] Negotiation successful
0806 19:34:41.773849 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":412,"threads_started":1}
I20260806 19:34:41.777558 12470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:41.746349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:41915 (local address 127.10.25.2:47169)
0806 19:34:41.747341 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:41.747391 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:41.749558 (+ 2167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:41.750488 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:41.750504 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:41.751479 (+ 975us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:41.752517 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:41.752556 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:41.754199 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:41.754203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:41.756787 (+ 2584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:41.756807 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:41.758062 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:41.758073 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:41.758321 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:41.759159 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:41.759186 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:41.761959 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:41.768229 (+ 6270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:41.768245 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:41.768275 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:41.769742 (+ 1467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:41.770441 (+ 699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:41.770450 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:41.770458 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:41.770944 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:41.771909 (+ 965us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:41.771923 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:41.772704 (+ 781us) client_negotiation.cc:769] Sending connection context
0806 19:34:41.773214 (+ 510us) client_negotiation.cc:240] Negotiation successful
0806 19:34:41.774208 (+ 994us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":442,"threads_started":1}
I20260806 19:34:41.809703 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:41.788752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:39895 (local address 127.10.25.62:41915)
0806 19:34:41.789005 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:41.789011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:41.789764 (+ 753us) server_negotiation.cc:407] Connection header received
0806 19:34:41.790094 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:41.790099 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:41.790184 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:41.790304 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:41.792176 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:41.793380 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:41.796436 (+ 3056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:41.797155 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:41.797295 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:41.801346 (+ 4051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:41.801353 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:41.801357 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:41.801400 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:41.806211 (+ 4811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:41.807178 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:41.807186 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:41.807191 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:41.807352 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:41.807807 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:41.807814 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:41.807817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:41.808241 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:41.808428 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:41.808965 (+ 537us) server_negotiation.cc:299] Negotiation successful
0806 19:34:41.809268 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 19:34:41.810197 12551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:41.788784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:41915 (local address 127.10.25.2:39895)
0806 19:34:41.789597 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:41.789634 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:41.789924 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:41.790348 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:41.790352 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:41.790888 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:41.791600 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:41.791635 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:41.793739 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:41.793747 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:41.796064 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:41.796099 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:41.797452 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:41.797462 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:41.797803 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:41.798971 (+ 1168us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:41.799044 (+ 73us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:41.800897 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:41.806440 (+ 5543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:41.806447 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:41.806452 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:41.807012 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:41.807492 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:41.807498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:41.807502 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:41.807675 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:41.808383 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:41.808389 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:41.808654 (+ 265us) client_negotiation.cc:769] Sending connection context
0806 19:34:41.808845 (+ 191us) client_negotiation.cc:240] Negotiation successful
0806 19:34:41.809295 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":424,"threads_started":1}
I20260806 19:34:41.817312 12445 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:34451
I20260806 19:34:41.817348 12589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:34451 every 8 connection(s)
I20260806 19:34:41.818893 12445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:34:41.822641 12590 heartbeater.cc:347] Connected to a master server at 127.10.25.62:41915
I20260806 19:34:41.823010 12590 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:41.823645 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 12445
I20260806 19:34:41.823851 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:34:41.824184 12590 heartbeater.cc:511] Master 127.10.25.62:41915 requested a full tablet report, sending...
I20260806 19:34:41.824594 10340 external_mini_cluster.cc:1428] Setting key e3db793af9b52765b1338cd897d716b3
I20260806 19:34:41.826928 12225 ts_manager.cc:194] Registered new tserver with Master: 575303e10edd446db43b7c363ecd132a (127.10.25.2:34451)
I20260806 19:34:41.827982 12225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:47169
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:34:41.886799 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:41915
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:37185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:34:42.190177 12440 heartbeater.cc:503] Master 127.10.25.62:41915 was elected leader, sending a full tablet report...
W20260806 19:34:42.305284 12594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:42.305660 12594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:42.305850 12594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:42.315058 12594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:42.315249 12594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:42.315428 12594 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 19:34:42.315536 12594 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:34:42.315685 12594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:34:42.327682 12594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:37185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.25.62:41915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:42.329348 12594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:42.330894 12594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:42.347409 12599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:42.347436 12600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:42.347337 12602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:42.348541 12594 server_base.cc:1034] running on GCE node
I20260806 19:34:42.349390 12594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:42.350767 12594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:42.352164 12594 hybrid_clock.cc:648] HybridClock initialized: now 1786044882352093 us; error 78 us; skew 500 ppm
I20260806 19:34:42.352676 12594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:42 dist-test-slave-ss4n krb5kdc[12175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044882, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:42.356670 12594 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:34:42.358948 12594 webserver.cc:460] Webserver started at http://127.10.25.3:46867/ using document root <none> and password file <none>
I20260806 19:34:42.359773 12594 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:42.360073 12594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:42.360723 12594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:42.365311 12594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "529f95f853204b94b5f6e1b86a0bc092"
format_stamp: "Formatted at 2026-08-06 19:34:42 on dist-test-slave-ss4n"
server_key: "8917262f7473c298cf27010fec00a82b"
server_key_iv: "86fd117ee2e42ae9d426ed7d48690b95"
server_key_version: "encryptionkey@0"
I20260806 19:34:42.366444 12594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "529f95f853204b94b5f6e1b86a0bc092"
format_stamp: "Formatted at 2026-08-06 19:34:42 on dist-test-slave-ss4n"
server_key: "8917262f7473c298cf27010fec00a82b"
server_key_iv: "86fd117ee2e42ae9d426ed7d48690b95"
server_key_version: "encryptionkey@0"
I20260806 19:34:42.373309 12594 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:34:42.378875 12609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:42.380682 12594 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260806 19:34:42.380935 12594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "529f95f853204b94b5f6e1b86a0bc092"
format_stamp: "Formatted at 2026-08-06 19:34:42 on dist-test-slave-ss4n"
server_key: "8917262f7473c298cf27010fec00a82b"
server_key_iv: "86fd117ee2e42ae9d426ed7d48690b95"
server_key_version: "encryptionkey@0"
I20260806 19:34:42.381325 12594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:42.423877 12594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:42.429534 12594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:42.429924 12594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:42.433024 12594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:42.433180 12594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:42.433324 12594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:42.433426 12594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:34:42 dist-test-slave-ss4n krb5kdc[12175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044882, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:42.464524 12618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.433959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:58829 (local address 127.10.25.62:41915)
0806 19:34:42.434700 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:42.434709 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:42.436231 (+ 1522us) server_negotiation.cc:407] Connection header received
0806 19:34:42.439722 (+ 3491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:42.439731 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:42.439849 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:42.439986 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:42.443241 (+ 3255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.444664 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.447773 (+ 3109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.448272 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.448444 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.455329 (+ 6885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:42.455335 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:42.455338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:42.455378 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:42.458680 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:42.460287 (+ 1607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:42.460297 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:42.460302 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:42.460502 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:42.461200 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:42.461208 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:42.461214 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:42.461803 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:42.462004 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:42.463236 (+ 1232us) server_negotiation.cc:299] Negotiation successful
0806 19:34:42.463728 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":439,"threads_started":1}
I20260806 19:34:42.464887 12619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.434863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:41915 (local address 127.10.25.3:58829)
0806 19:34:42.435976 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:42.436048 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:42.439244 (+ 3196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:42.440393 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:42.440410 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:42.441401 (+ 991us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:42.442931 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.442970 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.444994 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.445003 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.447485 (+ 2482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.447527 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.448727 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.448740 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.449166 (+ 426us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.451109 (+ 1943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:42.451166 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:42.454991 (+ 3825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:42.458927 (+ 3936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:42.458955 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:42.458985 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:42.460058 (+ 1073us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:42.460651 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:42.460664 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:42.460667 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:42.460982 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:42.461988 (+ 1006us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:42.462001 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:42.462575 (+ 574us) client_negotiation.cc:769] Sending connection context
0806 19:34:42.463010 (+ 435us) client_negotiation.cc:240] Negotiation successful
0806 19:34:42.463537 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":445,"threads_started":1}
I20260806 19:34:42.492589 12618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.472634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:46637 (local address 127.10.25.62:41915)
0806 19:34:42.472930 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:42.472939 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:42.473269 (+ 330us) server_negotiation.cc:407] Connection header received
0806 19:34:42.473457 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:42.473466 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:42.473639 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:42.473787 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:42.475153 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.476842 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.479206 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.479724 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.479896 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.483718 (+ 3822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:42.483731 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:42.483740 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:42.483811 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:42.488225 (+ 4414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:42.489728 (+ 1503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:42.489736 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:42.489739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:42.490027 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:42.490744 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:42.490750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:42.490752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:42.491146 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:42.491292 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:42.492051 (+ 759us) server_negotiation.cc:299] Negotiation successful
0806 19:34:42.492300 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260806 19:34:42.493213 12683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.472379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:41915 (local address 127.10.25.3:46637)
0806 19:34:42.473121 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:42.473143 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:42.473303 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:42.473891 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:42.473897 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:42.474509 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:42.474910 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.474922 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.477099 (+ 2177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.477105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.478982 (+ 1877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.479004 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.479992 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.480002 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.480306 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.481314 (+ 1008us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:42.481346 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:42.483286 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:42.488418 (+ 5132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:42.488429 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:42.488437 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:42.489322 (+ 885us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:42.490248 (+ 926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:42.490258 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:42.490265 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:42.490553 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:42.491323 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:42.491332 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:42.491674 (+ 342us) client_negotiation.cc:769] Sending connection context
0806 19:34:42.491907 (+ 233us) client_negotiation.cc:240] Negotiation successful
0806 19:34:42.492474 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":420,"threads_started":1}
I20260806 19:34:42.506644 12594 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:33861
I20260806 19:34:42.506690 12734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:33861 every 8 connection(s)
I20260806 19:34:42.508271 12594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:34:42.511801 12735 heartbeater.cc:347] Connected to a master server at 127.10.25.62:41915
I20260806 19:34:42.512121 12735 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:42.512925 12735 heartbeater.cc:511] Master 127.10.25.62:41915 requested a full tablet report, sending...
I20260806 19:34:42.514746 12225 ts_manager.cc:194] Registered new tserver with Master: 529f95f853204b94b5f6e1b86a0bc092 (127.10.25.3:33861)
I20260806 19:34:42.515566 12225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:58829
I20260806 19:34:42.516461 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 12594
I20260806 19:34:42.516676 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:34:42.517151 10340 external_mini_cluster.cc:1428] Setting key a33d0c055e59e8b2e50d2b25c62a8201
I20260806 19:34:42.521678 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 19:34:42 dist-test-slave-ss4n krb5kdc[12175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 19:34:42 dist-test-slave-ss4n krb5kdc[12175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:42.586906 12618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.567696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52194 (local address 127.10.25.62:41915)
0806 19:34:42.567976 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:42.567984 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:42.568296 (+ 312us) server_negotiation.cc:407] Connection header received
0806 19:34:42.568488 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:42.568492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:42.568627 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:42.568794 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:42.570445 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.571270 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.572722 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.573207 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.573349 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.578900 (+ 5551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:42.578906 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:42.578910 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:42.578959 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:42.583177 (+ 4218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:42.584121 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:42.584127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:42.584130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:42.584297 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:42.585061 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:42.585065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:42.585068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:42.585466 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:42.585940 (+ 474us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:42.586323 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 19:34:42.586525 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 19:34:42.593101 12225 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52194:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:34:42.601042 12225 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:34:42.632712 12753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.618971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36820 (local address 127.10.25.1:43133)
0806 19:34:42.620565 (+ 1594us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:42.620573 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:42.621969 (+ 1396us) server_negotiation.cc:407] Connection header received
0806 19:34:42.622368 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:42.622379 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:42.622725 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:42.622906 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:42.623721 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.625909 (+ 2188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.629298 (+ 3389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.631032 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.631192 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.631463 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:42.632252 (+ 789us) server_negotiation.cc:299] Negotiation successful
0806 19:34:42.632433 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1414,"thread_start_us":1272,"threads_started":1}
I20260806 19:34:42.632849 12752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.618500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:43133 (local address 127.0.0.1:36820)
0806 19:34:42.621832 (+ 3332us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:42.621860 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:42.622109 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:42.623170 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:42.623176 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:42.623202 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:42.623535 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.623545 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.626094 (+ 2549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.626100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.629088 (+ 2988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.629103 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.631641 (+ 2538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.631646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.631944 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.632012 (+ 68us) client_negotiation.cc:769] Sending connection context
0806 19:34:42.632199 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 19:34:42.632436 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3160,"thread_start_us":2978,"threads_started":1}
I20260806 19:34:42.634871 12755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.623018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:33861 (local address 127.0.0.1:56290)
0806 19:34:42.624890 (+ 1872us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:42.624920 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:42.625171 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:42.626414 (+ 1243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:42.626418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:42.626456 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:42.627200 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.627211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.628740 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.628748 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.631569 (+ 2821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.631587 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.633896 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.633902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.634078 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.634146 (+ 68us) client_negotiation.cc:769] Sending connection context
0806 19:34:42.634252 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 19:34:42.634458 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1706,"mutex_wait_us":190,"thread_start_us":1550,"threads_started":1}
I20260806 19:34:42.634958 12754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.622529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56290 (local address 127.10.25.3:33861)
0806 19:34:42.624366 (+ 1837us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:42.624377 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:42.625042 (+ 665us) server_negotiation.cc:407] Connection header received
0806 19:34:42.625446 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:42.625457 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:42.625786 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:42.626047 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:42.627432 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.628551 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.631800 (+ 3249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.633660 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.633821 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.634094 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:42.634373 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 19:34:42.634575 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1610,"thread_start_us":1348,"threads_started":1}
I20260806 19:34:42.636577 12750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.618308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:34451 (local address 127.0.0.1:42188)
0806 19:34:42.619541 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:42.619589 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:42.619838 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:42.626042 (+ 6204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:42.626047 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:42.626073 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:42.626717 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.626733 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.628877 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.628884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.631577 (+ 2693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.631587 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.635154 (+ 3567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.635162 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.635786 (+ 624us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.635827 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 19:34:42.635925 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 19:34:42.636082 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":951,"mutex_wait_us":116,"thread_start_us":825,"threads_started":1}
I20260806 19:34:42.637751 12751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.618518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42188 (local address 127.10.25.2:34451)
0806 19:34:42.625332 (+ 6814us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:42.625343 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:42.625375 (+ 32us) server_negotiation.cc:407] Connection header received
0806 19:34:42.625512 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:42.625522 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:42.625806 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:42.626754 (+ 948us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:42.626963 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.628663 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.633171 (+ 4508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.634879 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.636999 (+ 2120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.637188 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:42.637255 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 19:34:42.637395 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6648,"thread_start_us":6504,"threads_started":1}
I20260806 19:34:42.637693 12366 tablet_service.cc:1467] Processing CreateTablet for tablet d3a9ed8639b045c4826fdcd908239693 (DEFAULT_TABLE table=test-table [id=87a64ee6030748a79c32a0ebe51923be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:42.639348 12366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3a9ed8639b045c4826fdcd908239693. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:42.639624 12665 tablet_service.cc:1467] Processing CreateTablet for tablet d3a9ed8639b045c4826fdcd908239693 (DEFAULT_TABLE table=test-table [id=87a64ee6030748a79c32a0ebe51923be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:42.641197 12665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3a9ed8639b045c4826fdcd908239693. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:42.641175 12515 tablet_service.cc:1467] Processing CreateTablet for tablet d3a9ed8639b045c4826fdcd908239693 (DEFAULT_TABLE table=test-table [id=87a64ee6030748a79c32a0ebe51923be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:42.642652 12515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3a9ed8639b045c4826fdcd908239693. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:42.656198 12756 tablet_bootstrap.cc:492] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a: Bootstrap starting.
I20260806 19:34:42.660137 12756 tablet_bootstrap.cc:654] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:42.661901 12756 log.cc:824] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:34:42.663120 12757 tablet_bootstrap.cc:492] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630: Bootstrap starting.
I20260806 19:34:42.664846 12759 tablet_bootstrap.cc:492] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092: Bootstrap starting.
I20260806 19:34:42.666380 12756 tablet_bootstrap.cc:492] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a: No bootstrap required, opened a new log
I20260806 19:34:42.666769 12756 ts_tablet_manager.cc:1397] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260806 19:34:42.667902 12757 tablet_bootstrap.cc:654] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:42.669130 12759 tablet_bootstrap.cc:654] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:42.669553 12757 log.cc:824] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630: Log is configured to *not* fsync() on all Append() calls
I20260806 19:34:42.670603 12759 log.cc:824] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092: Log is configured to *not* fsync() on all Append() calls
I20260806 19:34:42.673424 12757 tablet_bootstrap.cc:492] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630: No bootstrap required, opened a new log
I20260806 19:34:42.673887 12757 ts_tablet_manager.cc:1397] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260806 19:34:42.674194 12759 tablet_bootstrap.cc:492] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092: No bootstrap required, opened a new log
I20260806 19:34:42.674523 12759 ts_tablet_manager.cc:1397] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260806 19:34:42.680645 12756 raft_consensus.cc:348] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:42.681316 12756 raft_consensus.cc:374] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:42.681437 12756 raft_consensus.cc:729] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575303e10edd446db43b7c363ecd132a, State: Initialized, Role: FOLLOWER
I20260806 19:34:42.682375 12756 consensus_queue.cc:260] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:42.684161 12590 heartbeater.cc:503] Master 127.10.25.62:41915 was elected leader, sending a full tablet report...
I20260806 19:34:42.685103 12756 ts_tablet_manager.cc:1428] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
I20260806 19:34:42.685117 12757 raft_consensus.cc:348] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:42.685722 12757 raft_consensus.cc:374] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:42.685806 12757 raft_consensus.cc:729] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0da279763745aa9289e16bc43ef630, State: Initialized, Role: FOLLOWER
I20260806 19:34:42.686784 12757 consensus_queue.cc:260] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:42.687115 12759 raft_consensus.cc:348] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:42.687726 12759 raft_consensus.cc:374] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:42.687860 12759 raft_consensus.cc:729] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 529f95f853204b94b5f6e1b86a0bc092, State: Initialized, Role: FOLLOWER
I20260806 19:34:42.688617 12759 consensus_queue.cc:260] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:42.690073 12735 heartbeater.cc:503] Master 127.10.25.62:41915 was elected leader, sending a full tablet report...
I20260806 19:34:42.690917 12759 ts_tablet_manager.cc:1428] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092: Time spent starting tablet: real 0.016s user 0.007s sys 0.009s
I20260806 19:34:42.691267 12757 ts_tablet_manager.cc:1428] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630: Time spent starting tablet: real 0.017s user 0.010s sys 0.004s
W20260806 19:34:42.760872 12737 tablet.cc:2369] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:34:42.822290 12591 tablet.cc:2369] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:34:42.836414 12766 raft_consensus.cc:482] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:34:42.836922 12766 raft_consensus.cc:504] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:42.839169 12766 leader_election.cc:288] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575303e10edd446db43b7c363ecd132a (127.10.25.2:34451), 3f0da279763745aa9289e16bc43ef630 (127.10.25.1:43133)
I20260806 19:34:42.844635 12765 raft_consensus.cc:482] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:34:42.845052 12765 raft_consensus.cc:504] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:42.846379 12619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.839590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:34451 (local address 127.10.25.3:56233)
0806 19:34:42.839780 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:42.839804 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:42.839928 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:42.840452 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:42.840458 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:42.840492 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:42.840744 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.840757 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.842034 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.842040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.843759 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.843774 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.845552 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.845559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.845772 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.845812 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 19:34:42.845922 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 19:34:42.846083 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 19:34:42.846451 12751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.839925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:56233 (local address 127.10.25.2:34451)
0806 19:34:42.840121 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:42.840126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:42.840146 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:34:42.840218 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:42.840222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:42.840285 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:42.840422 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:42.840921 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.841808 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.843950 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.845290 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.845451 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.845622 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:42.845981 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 19:34:42.846147 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 19:34:42.847390 12753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.839899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:37017 (local address 127.10.25.1:43133)
0806 19:34:42.840154 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:42.840161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:42.841021 (+ 860us) server_negotiation.cc:407] Connection header received
0806 19:34:42.841217 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:42.841224 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:42.841305 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:42.841404 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:42.842012 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.842769 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.844558 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.846079 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.846253 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.846375 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:42.846984 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 19:34:42.847139 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 19:34:42.847575 12767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.839610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:43133 (local address 127.10.25.3:37017)
0806 19:34:42.840895 (+ 1285us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:42.840918 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:42.841074 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:42.841433 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:42.841438 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:42.841470 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:42.841829 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.841839 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.842918 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.842928 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.844337 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.844349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.846546 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.846553 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.846774 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.846821 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 19:34:42.846942 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 19:34:42.847127 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1176,"mutex_wait_us":467,"thread_start_us":476,"threads_started":1}
I20260806 19:34:42.847355 12765 leader_election.cc:288] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575303e10edd446db43b7c363ecd132a (127.10.25.2:34451), 529f95f853204b94b5f6e1b86a0bc092 (127.10.25.3:33861)
I20260806 19:34:42.847934 12541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a9ed8639b045c4826fdcd908239693" candidate_uuid: "529f95f853204b94b5f6e1b86a0bc092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575303e10edd446db43b7c363ecd132a" is_pre_election: true
I20260806 19:34:42.848623 12541 raft_consensus.cc:2454] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 529f95f853204b94b5f6e1b86a0bc092 in term 0.
I20260806 19:34:42.849650 12612 leader_election.cc:302] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 529f95f853204b94b5f6e1b86a0bc092, 575303e10edd446db43b7c363ecd132a; no voters:
I20260806 19:34:42.850193 12766 raft_consensus.cc:2781] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:34:42.850369 12766 raft_consensus.cc:482] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:34:42.850498 12766 raft_consensus.cc:3037] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:42.850327 12395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a9ed8639b045c4826fdcd908239693" candidate_uuid: "529f95f853204b94b5f6e1b86a0bc092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0da279763745aa9289e16bc43ef630" is_pre_election: true
I20260806 19:34:42.850929 12395 raft_consensus.cc:2454] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 529f95f853204b94b5f6e1b86a0bc092 in term 0.
I20260806 19:34:42.854425 12766 raft_consensus.cc:504] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:42.855976 12541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a9ed8639b045c4826fdcd908239693" candidate_uuid: "529f95f853204b94b5f6e1b86a0bc092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575303e10edd446db43b7c363ecd132a"
I20260806 19:34:42.856249 12751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.848952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:51389 (local address 127.10.25.2:34451)
0806 19:34:42.849187 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:42.849195 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:42.849217 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:34:42.849287 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:42.849292 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:42.849370 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:42.850342 (+ 972us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:42.850499 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.851692 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.854032 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.854960 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.855114 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.855241 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:42.855694 (+ 453us) server_negotiation.cc:299] Negotiation successful
0806 19:34:42.855830 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 19:34:42.856303 12541 raft_consensus.cc:3037] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:42.856567 12768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.847791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:34451 (local address 127.10.25.1:51389)
0806 19:34:42.848646 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:42.848667 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:42.848896 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:42.849558 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:42.849567 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:42.849605 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:42.850193 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.850208 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.851859 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.851865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.853841 (+ 1976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.853855 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.855231 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.855239 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.855450 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.855489 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 19:34:42.855589 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 19:34:42.855754 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":1,"spinlock_wait_cycles":26880,"thread_start_us":412,"threads_started":1}
I20260806 19:34:42.857602 12395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a9ed8639b045c4826fdcd908239693" candidate_uuid: "529f95f853204b94b5f6e1b86a0bc092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0da279763745aa9289e16bc43ef630"
I20260806 19:34:42.857566 12537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a9ed8639b045c4826fdcd908239693" candidate_uuid: "3f0da279763745aa9289e16bc43ef630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575303e10edd446db43b7c363ecd132a" is_pre_election: true
I20260806 19:34:42.857860 12395 raft_consensus.cc:3037] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:42.858238 12754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.848581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:59603 (local address 127.10.25.3:33861)
0806 19:34:42.848784 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:42.848790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:42.849606 (+ 816us) server_negotiation.cc:407] Connection header received
0806 19:34:42.849878 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:42.849886 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:42.849966 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:42.850040 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:34:42.851349 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.852570 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.855382 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.856815 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.856948 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.857087 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:42.857803 (+ 716us) server_negotiation.cc:299] Negotiation successful
0806 19:34:42.857965 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 19:34:42.858987 12769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.848415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:33861 (local address 127.10.25.1:59603)
0806 19:34:42.849516 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:42.849537 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:42.849702 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:42.850415 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:42.850420 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:42.850515 (+ 95us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:34:42.851088 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.851103 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.852739 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.852745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.855151 (+ 2406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.855170 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.857309 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.857318 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.857560 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.857601 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 19:34:42.858414 (+ 813us) client_negotiation.cc:240] Negotiation successful
0806 19:34:42.858591 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"thread_start_us":753,"threads_started":1}
I20260806 19:34:42.859418 12766 leader_election.cc:288] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [CANDIDATE]: Term 1 election: Requested vote from peers 575303e10edd446db43b7c363ecd132a (127.10.25.2:34451), 3f0da279763745aa9289e16bc43ef630 (127.10.25.1:43133)
I20260806 19:34:42.860364 12541 raft_consensus.cc:2454] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 529f95f853204b94b5f6e1b86a0bc092 in term 1.
I20260806 19:34:42.860392 12690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a9ed8639b045c4826fdcd908239693" candidate_uuid: "3f0da279763745aa9289e16bc43ef630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "529f95f853204b94b5f6e1b86a0bc092" is_pre_election: true
I20260806 19:34:42.860957 12612 leader_election.cc:302] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 529f95f853204b94b5f6e1b86a0bc092, 575303e10edd446db43b7c363ecd132a; no voters:
I20260806 19:34:42.861104 12690 raft_consensus.cc:2379] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f0da279763745aa9289e16bc43ef630 in current term 1: Already voted for candidate 529f95f853204b94b5f6e1b86a0bc092 in this term.
I20260806 19:34:42.861411 12766 raft_consensus.cc:2781] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:34:42.861830 12395 raft_consensus.cc:2454] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 529f95f853204b94b5f6e1b86a0bc092 in term 1.
I20260806 19:34:42.862251 12766 raft_consensus.cc:686] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 LEADER]: Becoming Leader. State: Replica: 529f95f853204b94b5f6e1b86a0bc092, State: Running, Role: LEADER
I20260806 19:34:42.862213 12318 leader_election.cc:302] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f0da279763745aa9289e16bc43ef630; no voters: 529f95f853204b94b5f6e1b86a0bc092, 575303e10edd446db43b7c363ecd132a
I20260806 19:34:42.862680 12765 raft_consensus.cc:2726] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 19:34:42.862977 12766 consensus_queue.cc:237] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:42.870895 12225 catalog_manager.cc:5495] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 reported cstate change: term changed from 0 to 1, leader changed from <none> to 529f95f853204b94b5f6e1b86a0bc092 (127.10.25.3). New cstate: current_term: 1 leader_uuid: "529f95f853204b94b5f6e1b86a0bc092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } health_report { overall_health: HEALTHY } } }
W20260806 19:34:42.929811 12441 tablet.cc:2369] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 19:34:42 dist-test-slave-ss4n krb5kdc[12175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044880, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:42.975421 12618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.960504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52196 (local address 127.10.25.62:41915)
0806 19:34:42.960699 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:42.960708 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:42.961314 (+ 606us) server_negotiation.cc:407] Connection header received
0806 19:34:42.961565 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:42.961569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:42.961624 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:42.961731 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:42.962994 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.963823 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.965164 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.965669 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.965817 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.969538 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:42.969544 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:42.969547 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:42.969581 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:42.972323 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:42.973104 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:42.973113 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:42.973118 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:42.973277 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:42.973749 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:42.973756 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:42.973759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:42.974044 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:42.974205 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:42.974649 (+ 444us) server_negotiation.cc:299] Negotiation successful
0806 19:34:42.975049 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":163,"server-negotiator.queue_time_us":74}
I20260806 19:34:42.975455 12777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:42.960496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:41915 (local address 127.0.0.1:52196)
0806 19:34:42.961191 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:42.961209 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:42.961401 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:42.961767 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:42.961771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:42.962533 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:42.962831 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.962841 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.963955 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.963963 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.964989 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:42.965002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:42.965868 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:42.965874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:42.966045 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:42.966817 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:42.966836 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:42.969294 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:42.972488 (+ 3194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:42.972505 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:42.972514 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:42.972975 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:42.973403 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:42.973407 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:42.973411 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:42.973633 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:42.974189 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:42.974198 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:42.974379 (+ 181us) client_negotiation.cc:769] Sending connection context
0806 19:34:42.974550 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 19:34:42.974923 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":5,"thread_start_us":368,"threads_started":1}
I20260806 19:34:42.981428 12225 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:52196:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 19:34:42.982630 12225 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:34:42.991804 12366 tablet_service.cc:1467] Processing CreateTablet for tablet 58b88240745d446280cadbc4da455ea9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8310ebb87e7944d6ab90c8b1b38cba42]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:34:42.991876 12665 tablet_service.cc:1467] Processing CreateTablet for tablet 58b88240745d446280cadbc4da455ea9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8310ebb87e7944d6ab90c8b1b38cba42]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:34:42.991806 12515 tablet_service.cc:1467] Processing CreateTablet for tablet 58b88240745d446280cadbc4da455ea9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8310ebb87e7944d6ab90c8b1b38cba42]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:34:42.992810 12366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58b88240745d446280cadbc4da455ea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:42.992853 12665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58b88240745d446280cadbc4da455ea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:42.992853 12515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58b88240745d446280cadbc4da455ea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:43.000429 12757 tablet_bootstrap.cc:492] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630: Bootstrap starting.
I20260806 19:34:43.001241 12759 tablet_bootstrap.cc:492] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092: Bootstrap starting.
I20260806 19:34:43.002473 12757 tablet_bootstrap.cc:654] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:43.003198 12759 tablet_bootstrap.cc:654] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:43.004720 12756 tablet_bootstrap.cc:492] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a: Bootstrap starting.
I20260806 19:34:43.007112 12759 tablet_bootstrap.cc:492] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092: No bootstrap required, opened a new log
I20260806 19:34:43.007098 12756 tablet_bootstrap.cc:654] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:43.007298 12759 ts_tablet_manager.cc:1397] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20260806 19:34:43.008400 12759 raft_consensus.cc:348] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.008975 12757 tablet_bootstrap.cc:492] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630: No bootstrap required, opened a new log
I20260806 19:34:43.009029 12759 raft_consensus.cc:374] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:43.009197 12757 ts_tablet_manager.cc:1397] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260806 19:34:43.009181 12759 raft_consensus.cc:729] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 529f95f853204b94b5f6e1b86a0bc092, State: Initialized, Role: FOLLOWER
I20260806 19:34:43.009613 12759 consensus_queue.cc:260] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.010210 12757 raft_consensus.cc:348] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.010515 12757 raft_consensus.cc:374] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:43.010370 12766 tablet_replica.cc:439] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } } }
I20260806 19:34:43.010620 12757 raft_consensus.cc:729] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0da279763745aa9289e16bc43ef630, State: Initialized, Role: FOLLOWER
I20260806 19:34:43.010639 12759 ts_tablet_manager.cc:1428] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 19:34:43.010761 12766 tablet_replica.cc:442] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:34:43.010939 12757 consensus_queue.cc:260] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.011667 12757 ts_tablet_manager.cc:1428] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 19:34:43.011603 12765 tablet_replica.cc:439] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } } }
I20260806 19:34:43.012028 12765 tablet_replica.cc:442] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:34:43.013981 12756 tablet_bootstrap.cc:492] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a: No bootstrap required, opened a new log
I20260806 19:34:43.014246 12756 ts_tablet_manager.cc:1397] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260806 19:34:43.015389 12756 raft_consensus.cc:348] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.015623 12756 raft_consensus.cc:374] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:43.015744 12756 raft_consensus.cc:729] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575303e10edd446db43b7c363ecd132a, State: Initialized, Role: FOLLOWER
I20260806 19:34:43.015986 12756 consensus_queue.cc:260] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.016737 12756 ts_tablet_manager.cc:1428] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 19:34:43.016579 12764 tablet_replica.cc:439] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } } }
I20260806 19:34:43.017107 12764 tablet_replica.cc:442] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:34:43.257638 12766 raft_consensus.cc:482] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:34:43.257889 12766 raft_consensus.cc:504] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.258648 12766 leader_election.cc:288] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f0da279763745aa9289e16bc43ef630 (127.10.25.1:43133), 575303e10edd446db43b7c363ecd132a (127.10.25.2:34451)
I20260806 19:34:43.259320 12541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58b88240745d446280cadbc4da455ea9" candidate_uuid: "529f95f853204b94b5f6e1b86a0bc092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575303e10edd446db43b7c363ecd132a" is_pre_election: true
I20260806 19:34:43.259338 12395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58b88240745d446280cadbc4da455ea9" candidate_uuid: "529f95f853204b94b5f6e1b86a0bc092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0da279763745aa9289e16bc43ef630" is_pre_election: true
I20260806 19:34:43.259617 12541 raft_consensus.cc:2454] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 529f95f853204b94b5f6e1b86a0bc092 in term 0.
I20260806 19:34:43.259624 12395 raft_consensus.cc:2454] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 529f95f853204b94b5f6e1b86a0bc092 in term 0.
I20260806 19:34:43.260149 12612 leader_election.cc:302] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3f0da279763745aa9289e16bc43ef630, 529f95f853204b94b5f6e1b86a0bc092, 575303e10edd446db43b7c363ecd132a; no voters:
I20260806 19:34:43.260473 12766 raft_consensus.cc:2781] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:34:43.260571 12766 raft_consensus.cc:482] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:34:43.260696 12766 raft_consensus.cc:3037] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:43.262853 12766 raft_consensus.cc:504] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.263615 12766 leader_election.cc:288] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [CANDIDATE]: Term 1 election: Requested vote from peers 3f0da279763745aa9289e16bc43ef630 (127.10.25.1:43133), 575303e10edd446db43b7c363ecd132a (127.10.25.2:34451)
I20260806 19:34:43.264174 12541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58b88240745d446280cadbc4da455ea9" candidate_uuid: "529f95f853204b94b5f6e1b86a0bc092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575303e10edd446db43b7c363ecd132a"
I20260806 19:34:43.264146 12395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58b88240745d446280cadbc4da455ea9" candidate_uuid: "529f95f853204b94b5f6e1b86a0bc092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0da279763745aa9289e16bc43ef630"
I20260806 19:34:43.264460 12541 raft_consensus.cc:3037] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:43.264488 12395 raft_consensus.cc:3037] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:43.266676 12541 raft_consensus.cc:2454] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 529f95f853204b94b5f6e1b86a0bc092 in term 1.
I20260806 19:34:43.266758 12395 raft_consensus.cc:2454] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 529f95f853204b94b5f6e1b86a0bc092 in term 1.
I20260806 19:34:43.267268 12613 leader_election.cc:302] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3f0da279763745aa9289e16bc43ef630, 529f95f853204b94b5f6e1b86a0bc092, 575303e10edd446db43b7c363ecd132a; no voters:
I20260806 19:34:43.267709 12766 raft_consensus.cc:2781] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:34:43.267956 12766 raft_consensus.cc:686] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 LEADER]: Becoming Leader. State: Replica: 529f95f853204b94b5f6e1b86a0bc092, State: Running, Role: LEADER
I20260806 19:34:43.268229 12766 consensus_queue.cc:237] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.269934 12770 tablet_replica.cc:439] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092: TxnStatusTablet state changed. Reason: New leader 529f95f853204b94b5f6e1b86a0bc092. Latest consensus state: current_term: 1 leader_uuid: "529f95f853204b94b5f6e1b86a0bc092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } } }
I20260806 19:34:43.270226 12770 tablet_replica.cc:442] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:34:43.271589 12224 catalog_manager.cc:5495] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 reported cstate change: term changed from 0 to 1, leader changed from <none> to 529f95f853204b94b5f6e1b86a0bc092 (127.10.25.3). New cstate: current_term: 1 leader_uuid: "529f95f853204b94b5f6e1b86a0bc092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } health_report { overall_health: HEALTHY } } }
I20260806 19:34:43.273993 12782 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 19:34:43.274323 12782 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 19:34:43.314486 12777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:43.307972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:33861 (local address 127.0.0.1:56292)
0806 19:34:43.308282 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:43.308310 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:43.308524 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:43.309126 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:43.309131 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:43.309144 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:34:43.309356 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:43.309365 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:43.310547 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:43.310553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:43.311601 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:43.311613 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:43.312406 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:43.312413 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:43.312595 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:43.312668 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:34:43.313835 (+ 1167us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:34:43.313864 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 19:34:43.313978 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 19:34:43.314162 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 19:34:43.314517 12754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:43.308099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56292 (local address 127.10.25.3:33861)
0806 19:34:43.308294 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:43.308302 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:43.308407 (+ 105us) server_negotiation.cc:407] Connection header received
0806 19:34:43.308778 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:43.308782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:43.308865 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:43.309090 (+ 225us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:34:43.309518 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:43.310384 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:43.311773 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:43.312219 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:43.312372 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:43.312903 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:34:43.313670 (+ 767us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:34:43.313876 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:43.314002 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 19:34:43.314107 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 19:34:43.323494 12541 raft_consensus.cc:1261] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Refusing update from remote peer 529f95f853204b94b5f6e1b86a0bc092: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:34:43.323496 12395 raft_consensus.cc:1261] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630 [term 1 FOLLOWER]: Refusing update from remote peer 529f95f853204b94b5f6e1b86a0bc092: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:34:43.324577 12770 consensus_queue.cc:1035] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:34:43.325150 12766 consensus_queue.cc:1035] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:34:43.337085 12765 tablet_replica.cc:439] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630: TxnStatusTablet state changed. Reason: New leader 529f95f853204b94b5f6e1b86a0bc092. Latest consensus state: current_term: 1 leader_uuid: "529f95f853204b94b5f6e1b86a0bc092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } } }
I20260806 19:34:43.337440 12765 tablet_replica.cc:442] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:34:43.344676 12787 mvcc.cc:205] Tried to move back new op lower bound from 7315639842081419264 to 7315639841869365248. Current Snapshot: MvccSnapshot[applied={T|T < 7315639842081419264}]
I20260806 19:34:43.346184 12785 mvcc.cc:205] Tried to move back new op lower bound from 7315639842081419264 to 7315639841869365248. Current Snapshot: MvccSnapshot[applied={T|T < 7315639842081419264}]
I20260806 19:34:43.347060 12766 tablet_replica.cc:439] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "529f95f853204b94b5f6e1b86a0bc092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } } }
I20260806 19:34:43.347429 12766 tablet_replica.cc:442] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:34:43.352154 12765 tablet_replica.cc:439] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "529f95f853204b94b5f6e1b86a0bc092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } } }
I20260806 19:34:43.352479 12765 tablet_replica.cc:442] T 58b88240745d446280cadbc4da455ea9 P 3f0da279763745aa9289e16bc43ef630: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:34:43.360375 12770 consensus_queue.cc:1035] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:34:43.363031 12764 tablet_replica.cc:439] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a: TxnStatusTablet state changed. Reason: New leader 529f95f853204b94b5f6e1b86a0bc092. Latest consensus state: current_term: 1 leader_uuid: "529f95f853204b94b5f6e1b86a0bc092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } } }
I20260806 19:34:43.363335 12764 tablet_replica.cc:442] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:34:43.366915 12770 tablet_replica.cc:439] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "529f95f853204b94b5f6e1b86a0bc092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } } }
I20260806 19:34:43.367259 12770 tablet_replica.cc:442] T 58b88240745d446280cadbc4da455ea9 P 529f95f853204b94b5f6e1b86a0bc092: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:34:43.372259 12786 mvcc.cc:205] Tried to move back new op lower bound from 7315639842081419264 to 7315639841869365248. Current Snapshot: MvccSnapshot[applied={T|T < 7315639842081419264}]
I20260806 19:34:43.386126 12764 tablet_replica.cc:439] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "529f95f853204b94b5f6e1b86a0bc092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } } }
I20260806 19:34:43.386539 12764 tablet_replica.cc:442] T 58b88240745d446280cadbc4da455ea9 P 575303e10edd446db43b7c363ecd132a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:34:43.396942 12766 consensus_queue.cc:1035] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:34:43.414444 12754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:43.401653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56306 (local address 127.10.25.3:33861)
0806 19:34:43.402379 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:43.402388 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:43.406904 (+ 4516us) server_negotiation.cc:407] Connection header received
0806 19:34:43.407341 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:43.407347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:43.407470 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:43.407592 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:34:43.408320 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:43.409441 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:43.411755 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:43.412433 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:43.412568 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:43.413340 (+ 772us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:34:43.413552 (+ 212us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:34:43.413668 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:43.413924 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 19:34:43.414108 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584}
I20260806 19:34:43.438475 12754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:43.430374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:50171 (local address 127.10.25.3:33861)
0806 19:34:43.430615 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:43.430620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:43.431295 (+ 675us) server_negotiation.cc:407] Connection header received
0806 19:34:43.431624 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:43.431629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:43.431703 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:43.431805 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:34:43.432432 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:43.433807 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:43.436048 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:43.436534 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:43.436728 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:43.437201 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:34:43.437615 (+ 414us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:34:43.437743 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:43.437899 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 19:34:43.438027 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":78}
I20260806 19:34:43.438588 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:43.430355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:33861 (local address 127.10.25.3:50171)
0806 19:34:43.431167 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:43.431185 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:43.431416 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:43.431831 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:43.431836 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:43.431848 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:34:43.432258 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:43.432268 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:43.434053 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:43.434060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:43.435810 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:43.435842 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:43.436746 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:43.436751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:43.436887 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:43.436953 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:34:43.437741 (+ 788us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:34:43.437754 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 19:34:43.437877 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 19:34:43.438098 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":64,"thread_start_us":421,"threads_started":1}
I20260806 19:34:43.540644 12224 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:34:43.541055 12224 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:34:43.546212 12224 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d: Sending DeleteTablet for 3 replicas of tablet d3a9ed8639b045c4826fdcd908239693
I20260806 19:34:43.547753 12515 tablet_service.cc:1514] Processing DeleteTablet for tablet d3a9ed8639b045c4826fdcd908239693 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:43 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:42188
I20260806 19:34:43.548101 12366 tablet_service.cc:1514] Processing DeleteTablet for tablet d3a9ed8639b045c4826fdcd908239693 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:43 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:36820
I20260806 19:34:43.549893 12664 tablet_service.cc:1514] Processing DeleteTablet for tablet d3a9ed8639b045c4826fdcd908239693 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:43 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:56290
I20260806 19:34:43.549844 12224 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52194:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:34:43.551081 12224 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:34:43.555418 12805 tablet_replica.cc:330] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630: stopping tablet replica
I20260806 19:34:43.556262 12806 tablet_replica.cc:330] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092: stopping tablet replica
I20260806 19:34:43.556473 12804 tablet_replica.cc:330] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a: stopping tablet replica
I20260806 19:34:43.556996 12805 raft_consensus.cc:2229] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:34:43.557004 12804 raft_consensus.cc:2229] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:34:43.557389 12804 raft_consensus.cc:2258] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:43.557636 12806 raft_consensus.cc:2229] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:34:43.558104 12805 raft_consensus.cc:2258] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:43.559939 12804 ts_tablet_manager.cc:1905] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:43.562953 12515 tablet_service.cc:1467] Processing CreateTablet for tablet 345adb384b07444d9c838b82f7757f6f (DEFAULT_TABLE table=test-table [id=1d1f8100f08c46f6b1ff109b4c1d0860]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:43.563635 12515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 345adb384b07444d9c838b82f7757f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:43.563402 12664 tablet_service.cc:1467] Processing CreateTablet for tablet 345adb384b07444d9c838b82f7757f6f (DEFAULT_TABLE table=test-table [id=1d1f8100f08c46f6b1ff109b4c1d0860]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:43.565083 12664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 345adb384b07444d9c838b82f7757f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:43.558899 12806 raft_consensus.cc:2258] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:43.561826 12366 tablet_service.cc:1467] Processing CreateTablet for tablet 345adb384b07444d9c838b82f7757f6f (DEFAULT_TABLE table=test-table [id=1d1f8100f08c46f6b1ff109b4c1d0860]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:43.569056 12366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 345adb384b07444d9c838b82f7757f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:43.569257 12806 ts_tablet_manager.cc:1905] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:43.578083 12805 ts_tablet_manager.cc:1905] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:43.582197 12807 tablet_bootstrap.cc:492] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092: Bootstrap starting.
I20260806 19:34:43.584484 12808 tablet_bootstrap.cc:492] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630: Bootstrap starting.
I20260806 19:34:43.585275 12807 tablet_bootstrap.cc:654] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:43.587393 12808 tablet_bootstrap.cc:654] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:43.590952 12807 tablet_bootstrap.cc:492] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092: No bootstrap required, opened a new log
I20260806 19:34:43.591185 12807 ts_tablet_manager.cc:1397] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 19:34:43.591274 12808 tablet_bootstrap.cc:492] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630: No bootstrap required, opened a new log
I20260806 19:34:43.591455 12808 ts_tablet_manager.cc:1397] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 19:34:43.592187 12807 raft_consensus.cc:348] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.592469 12807 raft_consensus.cc:374] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:43.592350 12808 raft_consensus.cc:348] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.592593 12807 raft_consensus.cc:729] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 529f95f853204b94b5f6e1b86a0bc092, State: Initialized, Role: FOLLOWER
I20260806 19:34:43.592649 12808 raft_consensus.cc:374] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:43.592753 12808 raft_consensus.cc:729] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0da279763745aa9289e16bc43ef630, State: Initialized, Role: FOLLOWER
I20260806 19:34:43.592962 12807 consensus_queue.cc:260] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.593030 12808 consensus_queue.cc:260] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.594729 12807 ts_tablet_manager.cc:1428] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 19:34:43.595862 12808 ts_tablet_manager.cc:1428] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 19:34:43.605165 12804 ts_tablet_manager.cc:1918] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:34:43.605413 12804 log.cc:1196] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/d3a9ed8639b045c4826fdcd908239693
I20260806 19:34:43.606097 12804 ts_tablet_manager.cc:1939] T d3a9ed8639b045c4826fdcd908239693 P 575303e10edd446db43b7c363ecd132a: Deleting consensus metadata
I20260806 19:34:43.609869 12811 tablet_bootstrap.cc:492] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a: Bootstrap starting.
I20260806 19:34:43.611824 12811 tablet_bootstrap.cc:654] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:43.614357 12811 tablet_bootstrap.cc:492] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a: No bootstrap required, opened a new log
I20260806 19:34:43.614574 12811 ts_tablet_manager.cc:1397] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260806 19:34:43.615464 12811 raft_consensus.cc:348] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.615720 12811 raft_consensus.cc:374] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:43.615816 12811 raft_consensus.cc:729] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575303e10edd446db43b7c363ecd132a, State: Initialized, Role: FOLLOWER
I20260806 19:34:43.616114 12811 consensus_queue.cc:260] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.616858 12811 ts_tablet_manager.cc:1428] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 19:34:43.663111 12766 raft_consensus.cc:482] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:34:43.663355 12766 raft_consensus.cc:504] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.664135 12766 leader_election.cc:288] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f0da279763745aa9289e16bc43ef630 (127.10.25.1:43133), 575303e10edd446db43b7c363ecd132a (127.10.25.2:34451)
I20260806 19:34:43.664853 12394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345adb384b07444d9c838b82f7757f6f" candidate_uuid: "529f95f853204b94b5f6e1b86a0bc092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0da279763745aa9289e16bc43ef630" is_pre_election: true
I20260806 19:34:43.664969 12537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345adb384b07444d9c838b82f7757f6f" candidate_uuid: "529f95f853204b94b5f6e1b86a0bc092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575303e10edd446db43b7c363ecd132a" is_pre_election: true
I20260806 19:34:43.665184 12394 raft_consensus.cc:2454] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 529f95f853204b94b5f6e1b86a0bc092 in term 0.
I20260806 19:34:43.665287 12537 raft_consensus.cc:2454] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 529f95f853204b94b5f6e1b86a0bc092 in term 0.
I20260806 19:34:43.665814 12613 leader_election.cc:302] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0da279763745aa9289e16bc43ef630, 529f95f853204b94b5f6e1b86a0bc092; no voters:
I20260806 19:34:43.666201 12766 raft_consensus.cc:2781] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:34:43.666350 12766 raft_consensus.cc:482] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:34:43.666505 12766 raft_consensus.cc:3037] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:43.678642 12210 catalog_manager.cc:4841] TS 575303e10edd446db43b7c363ecd132a (127.10.25.2:34451): tablet d3a9ed8639b045c4826fdcd908239693 (table test-table [id=87a64ee6030748a79c32a0ebe51923be]) successfully deleted
I20260806 19:34:43.706244 12806 ts_tablet_manager.cc:1918] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:34:43.706131 12766 raft_consensus.cc:504] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.706499 12806 log.cc:1196] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/d3a9ed8639b045c4826fdcd908239693
I20260806 19:34:43.707201 12806 ts_tablet_manager.cc:1939] T d3a9ed8639b045c4826fdcd908239693 P 529f95f853204b94b5f6e1b86a0bc092: Deleting consensus metadata
I20260806 19:34:43.707957 12537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345adb384b07444d9c838b82f7757f6f" candidate_uuid: "529f95f853204b94b5f6e1b86a0bc092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575303e10edd446db43b7c363ecd132a"
I20260806 19:34:43.708513 12537 raft_consensus.cc:3037] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:43.709995 12805 ts_tablet_manager.cc:1918] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:34:43.710207 12805 log.cc:1196] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/d3a9ed8639b045c4826fdcd908239693
I20260806 19:34:43.710765 12805 ts_tablet_manager.cc:1939] T d3a9ed8639b045c4826fdcd908239693 P 3f0da279763745aa9289e16bc43ef630: Deleting consensus metadata
I20260806 19:34:43.712195 12766 leader_election.cc:288] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [CANDIDATE]: Term 1 election: Requested vote from peers 3f0da279763745aa9289e16bc43ef630 (127.10.25.1:43133), 575303e10edd446db43b7c363ecd132a (127.10.25.2:34451)
I20260806 19:34:43.712320 12211 catalog_manager.cc:4841] TS 529f95f853204b94b5f6e1b86a0bc092 (127.10.25.3:33861): tablet d3a9ed8639b045c4826fdcd908239693 (table test-table [id=87a64ee6030748a79c32a0ebe51923be]) successfully deleted
I20260806 19:34:43.712013 12394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "345adb384b07444d9c838b82f7757f6f" candidate_uuid: "529f95f853204b94b5f6e1b86a0bc092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0da279763745aa9289e16bc43ef630"
I20260806 19:34:43.713431 12394 raft_consensus.cc:3037] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:43.779147 12211 catalog_manager.cc:4841] TS 3f0da279763745aa9289e16bc43ef630 (127.10.25.1:43133): tablet d3a9ed8639b045c4826fdcd908239693 (table test-table [id=87a64ee6030748a79c32a0ebe51923be]) successfully deleted
I20260806 19:34:43.806926 12537 raft_consensus.cc:2454] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 529f95f853204b94b5f6e1b86a0bc092 in term 1.
I20260806 19:34:43.807009 12394 raft_consensus.cc:2454] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 529f95f853204b94b5f6e1b86a0bc092 in term 1.
I20260806 19:34:43.807847 12612 leader_election.cc:302] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 529f95f853204b94b5f6e1b86a0bc092, 575303e10edd446db43b7c363ecd132a; no voters:
I20260806 19:34:43.808339 12766 raft_consensus.cc:2781] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:34:43.808614 12766 raft_consensus.cc:686] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [term 1 LEADER]: Becoming Leader. State: Replica: 529f95f853204b94b5f6e1b86a0bc092, State: Running, Role: LEADER
I20260806 19:34:43.808907 12766 consensus_queue.cc:237] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:43.812242 12225 catalog_manager.cc:5495] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 reported cstate change: term changed from 0 to 1, leader changed from <none> to 529f95f853204b94b5f6e1b86a0bc092 (127.10.25.3). New cstate: current_term: 1 leader_uuid: "529f95f853204b94b5f6e1b86a0bc092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } health_report { overall_health: HEALTHY } } }
I20260806 19:34:43.864688 12394 raft_consensus.cc:1261] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630 [term 1 FOLLOWER]: Refusing update from remote peer 529f95f853204b94b5f6e1b86a0bc092: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:34:43.864727 12537 raft_consensus.cc:1261] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Refusing update from remote peer 529f95f853204b94b5f6e1b86a0bc092: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:34:43.865353 12770 consensus_queue.cc:1035] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:34:43.865850 12795 consensus_queue.cc:1035] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 19:34:43.898648 12785 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 19:34:45.536895 12226 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:34:45.537258 12226 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:34:45.543295 12226 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d: Sending DeleteTablet for 3 replicas of tablet 345adb384b07444d9c838b82f7757f6f
I20260806 19:34:45.544399 12366 tablet_service.cc:1514] Processing DeleteTablet for tablet 345adb384b07444d9c838b82f7757f6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:45 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:36820
I20260806 19:34:45.544484 12664 tablet_service.cc:1514] Processing DeleteTablet for tablet 345adb384b07444d9c838b82f7757f6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:45 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:56290
I20260806 19:34:45.544509 12515 tablet_service.cc:1514] Processing DeleteTablet for tablet 345adb384b07444d9c838b82f7757f6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:45 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:42188
I20260806 19:34:45.555954 12838 tablet_replica.cc:330] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630: stopping tablet replica
I20260806 19:34:45.556298 12837 tablet_replica.cc:330] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092: stopping tablet replica
I20260806 19:34:45.556272 12838 raft_consensus.cc:2229] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:34:45.556624 12838 raft_consensus.cc:2258] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:45.556663 12837 raft_consensus.cc:2229] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:34:45.557331 12837 raft_consensus.cc:2258] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:45.557577 12839 tablet_replica.cc:330] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a: stopping tablet replica
I20260806 19:34:45.558240 12838 ts_tablet_manager.cc:1905] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:45.558310 12839 raft_consensus.cc:2229] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:34:45.558645 12839 raft_consensus.cc:2258] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:45.559054 12837 ts_tablet_manager.cc:1905] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:45.559909 12839 ts_tablet_manager.cc:1905] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:45.567031 12837 ts_tablet_manager.cc:1918] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 19:34:45.567162 12839 ts_tablet_manager.cc:1918] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 19:34:45.567312 12839 log.cc:1196] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/345adb384b07444d9c838b82f7757f6f
I20260806 19:34:45.567453 12837 log.cc:1196] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/345adb384b07444d9c838b82f7757f6f
I20260806 19:34:45.567826 12839 ts_tablet_manager.cc:1939] T 345adb384b07444d9c838b82f7757f6f P 575303e10edd446db43b7c363ecd132a: Deleting consensus metadata
I20260806 19:34:45.568074 12837 ts_tablet_manager.cc:1939] T 345adb384b07444d9c838b82f7757f6f P 529f95f853204b94b5f6e1b86a0bc092: Deleting consensus metadata
I20260806 19:34:45.569036 12838 ts_tablet_manager.cc:1918] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 19:34:45.569216 12838 log.cc:1196] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/345adb384b07444d9c838b82f7757f6f
I20260806 19:34:45.569228 12210 catalog_manager.cc:4841] TS 575303e10edd446db43b7c363ecd132a (127.10.25.2:34451): tablet 345adb384b07444d9c838b82f7757f6f (table test-table [id=1d1f8100f08c46f6b1ff109b4c1d0860]) successfully deleted
I20260806 19:34:45.569765 12211 catalog_manager.cc:4841] TS 529f95f853204b94b5f6e1b86a0bc092 (127.10.25.3:33861): tablet 345adb384b07444d9c838b82f7757f6f (table test-table [id=1d1f8100f08c46f6b1ff109b4c1d0860]) successfully deleted
I20260806 19:34:45.570068 12838 ts_tablet_manager.cc:1939] T 345adb384b07444d9c838b82f7757f6f P 3f0da279763745aa9289e16bc43ef630: Deleting consensus metadata
I20260806 19:34:45.571502 12211 catalog_manager.cc:4841] TS 3f0da279763745aa9289e16bc43ef630 (127.10.25.1:43133): tablet 345adb384b07444d9c838b82f7757f6f (table test-table [id=1d1f8100f08c46f6b1ff109b4c1d0860]) successfully deleted
Aug 06 19:34:45 dist-test-slave-ss4n krb5kdc[12175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044885, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 19:34:45 dist-test-slave-ss4n krb5kdc[12175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044885, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:45.626924 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:45.606091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52202 (local address 127.10.25.62:41915)
0806 19:34:45.606864 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:45.606869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:45.606898 (+ 29us) server_negotiation.cc:407] Connection header received
0806 19:34:45.607081 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:45.607085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:45.607164 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:45.607275 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:45.609432 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:45.610377 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:45.611852 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:45.612305 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:45.612462 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:45.618537 (+ 6075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:45.618544 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:45.618548 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:45.618607 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:45.622376 (+ 3769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:45.623877 (+ 1501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:45.623884 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:45.623887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:45.624149 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:45.625160 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:45.625166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:45.625168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:45.625724 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:45.625980 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:45.626368 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 19:34:45.626598 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":421,"threads_started":1}
I20260806 19:34:45.632519 12226 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:34:45.633837 12226 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:34:45.643313 12366 tablet_service.cc:1467] Processing CreateTablet for tablet 1b6d74d842fb4baf8a7e3764571da8e7 (DEFAULT_TABLE table=test-table [id=60b487698a414eb6acc4f27e509ead7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:45.643472 12515 tablet_service.cc:1467] Processing CreateTablet for tablet 1b6d74d842fb4baf8a7e3764571da8e7 (DEFAULT_TABLE table=test-table [id=60b487698a414eb6acc4f27e509ead7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:45.643952 12366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b6d74d842fb4baf8a7e3764571da8e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:45.643965 12515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b6d74d842fb4baf8a7e3764571da8e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:45.643970 12664 tablet_service.cc:1467] Processing CreateTablet for tablet 1b6d74d842fb4baf8a7e3764571da8e7 (DEFAULT_TABLE table=test-table [id=60b487698a414eb6acc4f27e509ead7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:45.644382 12664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b6d74d842fb4baf8a7e3764571da8e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:45.652009 12849 tablet_bootstrap.cc:492] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630: Bootstrap starting.
I20260806 19:34:45.652866 12850 tablet_bootstrap.cc:492] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092: Bootstrap starting.
I20260806 19:34:45.654007 12848 tablet_bootstrap.cc:492] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a: Bootstrap starting.
I20260806 19:34:45.655381 12850 tablet_bootstrap.cc:654] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:45.655845 12848 tablet_bootstrap.cc:654] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:45.655874 12849 tablet_bootstrap.cc:654] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:45.658567 12850 tablet_bootstrap.cc:492] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092: No bootstrap required, opened a new log
I20260806 19:34:45.658583 12848 tablet_bootstrap.cc:492] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a: No bootstrap required, opened a new log
I20260806 19:34:45.658797 12848 ts_tablet_manager.cc:1397] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 19:34:45.658787 12850 ts_tablet_manager.cc:1397] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 19:34:45.659871 12848 raft_consensus.cc:348] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:45.659873 12850 raft_consensus.cc:348] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:45.660117 12849 tablet_bootstrap.cc:492] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630: No bootstrap required, opened a new log
I20260806 19:34:45.660202 12850 raft_consensus.cc:374] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:45.660270 12848 raft_consensus.cc:374] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:45.660295 12849 ts_tablet_manager.cc:1397] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260806 19:34:45.660369 12850 raft_consensus.cc:729] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 529f95f853204b94b5f6e1b86a0bc092, State: Initialized, Role: FOLLOWER
I20260806 19:34:45.660406 12848 raft_consensus.cc:729] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575303e10edd446db43b7c363ecd132a, State: Initialized, Role: FOLLOWER
I20260806 19:34:45.660710 12848 consensus_queue.cc:260] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:45.660708 12850 consensus_queue.cc:260] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:45.661166 12849 raft_consensus.cc:348] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:45.661425 12849 raft_consensus.cc:374] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:45.661567 12849 raft_consensus.cc:729] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0da279763745aa9289e16bc43ef630, State: Initialized, Role: FOLLOWER
I20260806 19:34:45.661834 12850 ts_tablet_manager.cc:1428] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 19:34:45.661803 12849 consensus_queue.cc:260] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:45.663069 12848 ts_tablet_manager.cc:1428] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a: Time spent starting tablet: real 0.004s user 0.002s sys 0.003s
I20260806 19:34:45.664163 12849 ts_tablet_manager.cc:1428] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260806 19:34:45.828282 12855 raft_consensus.cc:482] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:34:45.828697 12855 raft_consensus.cc:504] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:45.830202 12855 leader_election.cc:288] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575303e10edd446db43b7c363ecd132a (127.10.25.2:34451), 529f95f853204b94b5f6e1b86a0bc092 (127.10.25.3:33861)
I20260806 19:34:45.831216 12690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6d74d842fb4baf8a7e3764571da8e7" candidate_uuid: "3f0da279763745aa9289e16bc43ef630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "529f95f853204b94b5f6e1b86a0bc092" is_pre_election: true
I20260806 19:34:45.831292 12537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6d74d842fb4baf8a7e3764571da8e7" candidate_uuid: "3f0da279763745aa9289e16bc43ef630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575303e10edd446db43b7c363ecd132a" is_pre_election: true
I20260806 19:34:45.831629 12690 raft_consensus.cc:2454] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0da279763745aa9289e16bc43ef630 in term 0.
I20260806 19:34:45.831744 12537 raft_consensus.cc:2454] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0da279763745aa9289e16bc43ef630 in term 0.
I20260806 19:34:45.832362 12318 leader_election.cc:302] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0da279763745aa9289e16bc43ef630, 529f95f853204b94b5f6e1b86a0bc092; no voters:
I20260806 19:34:45.832801 12855 raft_consensus.cc:2781] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:34:45.832967 12855 raft_consensus.cc:482] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:34:45.833102 12855 raft_consensus.cc:3037] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:45.836504 12855 raft_consensus.cc:504] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:45.837446 12855 leader_election.cc:288] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [CANDIDATE]: Term 1 election: Requested vote from peers 575303e10edd446db43b7c363ecd132a (127.10.25.2:34451), 529f95f853204b94b5f6e1b86a0bc092 (127.10.25.3:33861)
I20260806 19:34:45.838219 12537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6d74d842fb4baf8a7e3764571da8e7" candidate_uuid: "3f0da279763745aa9289e16bc43ef630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575303e10edd446db43b7c363ecd132a"
I20260806 19:34:45.838414 12690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6d74d842fb4baf8a7e3764571da8e7" candidate_uuid: "3f0da279763745aa9289e16bc43ef630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "529f95f853204b94b5f6e1b86a0bc092"
I20260806 19:34:45.838585 12537 raft_consensus.cc:3037] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:45.838738 12690 raft_consensus.cc:3037] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:45.842896 12537 raft_consensus.cc:2454] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0da279763745aa9289e16bc43ef630 in term 1.
I20260806 19:34:45.842926 12690 raft_consensus.cc:2454] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0da279763745aa9289e16bc43ef630 in term 1.
I20260806 19:34:45.843602 12317 leader_election.cc:302] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0da279763745aa9289e16bc43ef630, 575303e10edd446db43b7c363ecd132a; no voters:
I20260806 19:34:45.844020 12855 raft_consensus.cc:2781] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:34:45.844673 12855 raft_consensus.cc:686] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [term 1 LEADER]: Becoming Leader. State: Replica: 3f0da279763745aa9289e16bc43ef630, State: Running, Role: LEADER
I20260806 19:34:45.845377 12855 consensus_queue.cc:237] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:45.849896 12224 catalog_manager.cc:5495] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f0da279763745aa9289e16bc43ef630 (127.10.25.1). New cstate: current_term: 1 leader_uuid: "3f0da279763745aa9289e16bc43ef630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } health_report { overall_health: UNKNOWN } } }
I20260806 19:34:45.935956 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:45.914178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36832 (local address 127.10.25.1:43133)
0806 19:34:45.926054 (+ 11876us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:45.926063 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:45.926107 (+ 44us) server_negotiation.cc:407] Connection header received
0806 19:34:45.926256 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:45.926262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:45.926577 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:45.927068 (+ 491us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:34:45.927722 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:45.929280 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:45.932493 (+ 3213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:45.933203 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:45.933490 (+ 287us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:45.934189 (+ 699us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:34:45.935017 (+ 828us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:34:45.935187 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:45.935323 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 19:34:45.935419 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11628,"thread_start_us":11374,"threads_started":1}
I20260806 19:34:45.956192 12862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:45.949241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:33861 (local address 127.10.25.1:59659)
0806 19:34:45.950141 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:45.950160 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:45.950319 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:45.950978 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:45.950983 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:45.950995 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:34:45.951322 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:45.951330 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:45.952779 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:45.952785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:45.954050 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:45.954065 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:45.954934 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:45.954940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:45.955068 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:45.955135 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:34:45.955652 (+ 517us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:34:45.955662 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 19:34:45.955729 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 19:34:45.955872 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":525,"threads_started":1}
I20260806 19:34:45.956225 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:45.949288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:59659 (local address 127.10.25.3:33861)
0806 19:34:45.950090 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:45.950095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:45.950276 (+ 181us) server_negotiation.cc:407] Connection header received
0806 19:34:45.950587 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:45.950593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:45.950765 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:45.950914 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:34:45.951631 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:45.952587 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:45.954221 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:45.954744 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:45.954862 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:45.955368 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:34:45.955537 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:34:45.955622 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:45.955785 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 19:34:45.955866 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":478,"threads_started":1}
I20260806 19:34:45.967512 12537 raft_consensus.cc:1261] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Refusing update from remote peer 3f0da279763745aa9289e16bc43ef630: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:34:45.967674 12690 raft_consensus.cc:1261] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Refusing update from remote peer 3f0da279763745aa9289e16bc43ef630: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:34:45.968338 12856 consensus_queue.cc:1035] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:34:45.968782 12855 consensus_queue.cc:1035] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:34:46.007689 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:45.999048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:34299 (local address 127.10.25.1:43133)
0806 19:34:45.999240 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:45.999246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:45.999526 (+ 280us) server_negotiation.cc:407] Connection header received
0806 19:34:45.999643 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:45.999647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:45.999732 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:45.999833 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:34:46.000753 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:46.001880 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:46.004181 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:46.004945 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:46.005113 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:46.005916 (+ 803us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:34:46.006090 (+ 174us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:34:46.006195 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:46.006650 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 19:34:46.006776 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 19:34:46.007961 12870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:45.998768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:43133 (local address 127.10.25.3:34299)
0806 19:34:45.999378 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:45.999395 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:45.999529 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:45.999913 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:45.999917 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:45.999933 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:34:46.000456 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:46.000468 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:46.002096 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:46.002101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:46.003900 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:46.003916 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:46.005223 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:46.005229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:46.005398 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:46.005446 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:34:46.006356 (+ 910us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:34:46.006435 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 19:34:46.006603 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 19:34:46.006831 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":371,"threads_started":1}
I20260806 19:34:46.047192 12224 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:34:46.047750 12224 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:34:46.051690 12224 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d: Sending DeleteTablet for 3 replicas of tablet 1b6d74d842fb4baf8a7e3764571da8e7
I20260806 19:34:46.053474 12515 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b6d74d842fb4baf8a7e3764571da8e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:46 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:42188
I20260806 19:34:46.053897 12839 tablet_replica.cc:330] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a: stopping tablet replica
I20260806 19:34:46.054148 12839 raft_consensus.cc:2229] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:34:46.053861 12664 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b6d74d842fb4baf8a7e3764571da8e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:46 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:56290
I20260806 19:34:46.054395 12839 raft_consensus.cc:2258] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:46.054908 12837 tablet_replica.cc:330] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092: stopping tablet replica
I20260806 19:34:46.055065 12366 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b6d74d842fb4baf8a7e3764571da8e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:46 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:36820
I20260806 19:34:46.054893 12224 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 19:34:46.055171 12837 raft_consensus.cc:2229] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:34:46.055400 12837 raft_consensus.cc:2258] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 19:34:46.055655 12224 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:34:46.055751 12838 tablet_replica.cc:330] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630: stopping tablet replica
I20260806 19:34:46.056010 12838 raft_consensus.cc:2229] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:34:46.056391 12839 ts_tablet_manager.cc:1905] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:46.056617 12838 raft_consensus.cc:2258] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:46.059132 12837 ts_tablet_manager.cc:1905] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:46.059680 12838 ts_tablet_manager.cc:1905] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:46.064416 12839 ts_tablet_manager.cc:1918] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:34:46.064654 12839 log.cc:1196] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/1b6d74d842fb4baf8a7e3764571da8e7
I20260806 19:34:46.065296 12839 ts_tablet_manager.cc:1939] T 1b6d74d842fb4baf8a7e3764571da8e7 P 575303e10edd446db43b7c363ecd132a: Deleting consensus metadata
I20260806 19:34:46.068182 12210 catalog_manager.cc:4841] TS 575303e10edd446db43b7c363ecd132a (127.10.25.2:34451): tablet 1b6d74d842fb4baf8a7e3764571da8e7 (table test-table [id=60b487698a414eb6acc4f27e509ead7c]) successfully deleted
I20260806 19:34:46.068302 12837 ts_tablet_manager.cc:1918] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:34:46.068665 12838 ts_tablet_manager.cc:1918] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:34:46.068890 12838 log.cc:1196] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/1b6d74d842fb4baf8a7e3764571da8e7
I20260806 19:34:46.068915 12837 log.cc:1196] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/1b6d74d842fb4baf8a7e3764571da8e7
I20260806 19:34:46.069525 12837 ts_tablet_manager.cc:1939] T 1b6d74d842fb4baf8a7e3764571da8e7 P 529f95f853204b94b5f6e1b86a0bc092: Deleting consensus metadata
I20260806 19:34:46.070895 12838 ts_tablet_manager.cc:1939] T 1b6d74d842fb4baf8a7e3764571da8e7 P 3f0da279763745aa9289e16bc43ef630: Deleting consensus metadata
I20260806 19:34:46.071130 12211 catalog_manager.cc:4841] TS 529f95f853204b94b5f6e1b86a0bc092 (127.10.25.3:33861): tablet 1b6d74d842fb4baf8a7e3764571da8e7 (table test-table [id=60b487698a414eb6acc4f27e509ead7c]) successfully deleted
I20260806 19:34:46.072481 12211 catalog_manager.cc:4841] TS 3f0da279763745aa9289e16bc43ef630 (127.10.25.1:43133): tablet 1b6d74d842fb4baf8a7e3764571da8e7 (table test-table [id=60b487698a414eb6acc4f27e509ead7c]) successfully deleted
I20260806 19:34:46.075629 12366 tablet_service.cc:1467] Processing CreateTablet for tablet 435873710cec40e18ef804ad781a7bd7 (DEFAULT_TABLE table=test-table [id=96f17f6094d644358d54e8aa000f4155]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:46.076208 12366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 435873710cec40e18ef804ad781a7bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:46.076226 12664 tablet_service.cc:1467] Processing CreateTablet for tablet 435873710cec40e18ef804ad781a7bd7 (DEFAULT_TABLE table=test-table [id=96f17f6094d644358d54e8aa000f4155]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:46.076802 12664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 435873710cec40e18ef804ad781a7bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:46.082680 12515 tablet_service.cc:1467] Processing CreateTablet for tablet 435873710cec40e18ef804ad781a7bd7 (DEFAULT_TABLE table=test-table [id=96f17f6094d644358d54e8aa000f4155]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:34:46.083319 12515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 435873710cec40e18ef804ad781a7bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:46.085706 12849 tablet_bootstrap.cc:492] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630: Bootstrap starting.
I20260806 19:34:46.085804 12850 tablet_bootstrap.cc:492] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092: Bootstrap starting.
I20260806 19:34:46.088168 12849 tablet_bootstrap.cc:654] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:46.088234 12850 tablet_bootstrap.cc:654] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:46.096346 12848 tablet_bootstrap.cc:492] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a: Bootstrap starting.
I20260806 19:34:46.098925 12850 tablet_bootstrap.cc:492] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092: No bootstrap required, opened a new log
I20260806 19:34:46.099263 12850 ts_tablet_manager.cc:1397] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.007s
I20260806 19:34:46.099367 12848 tablet_bootstrap.cc:654] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:46.100472 12850 raft_consensus.cc:348] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:46.101053 12850 raft_consensus.cc:374] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:46.101193 12850 raft_consensus.cc:729] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 529f95f853204b94b5f6e1b86a0bc092, State: Initialized, Role: FOLLOWER
I20260806 19:34:46.101284 12849 tablet_bootstrap.cc:492] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630: No bootstrap required, opened a new log
I20260806 19:34:46.101730 12849 ts_tablet_manager.cc:1397] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.003s
I20260806 19:34:46.101698 12850 consensus_queue.cc:260] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:46.105700 12849 raft_consensus.cc:348] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:46.103336 12850 ts_tablet_manager.cc:1428] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260806 19:34:46.106089 12849 raft_consensus.cc:374] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:46.106444 12849 raft_consensus.cc:729] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0da279763745aa9289e16bc43ef630, State: Initialized, Role: FOLLOWER
I20260806 19:34:46.106787 12849 consensus_queue.cc:260] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:46.107625 12849 ts_tablet_manager.cc:1428] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260806 19:34:46.110796 12848 tablet_bootstrap.cc:492] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a: No bootstrap required, opened a new log
I20260806 19:34:46.111047 12848 ts_tablet_manager.cc:1397] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260806 19:34:46.112035 12848 raft_consensus.cc:348] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:46.112357 12848 raft_consensus.cc:374] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:46.112454 12848 raft_consensus.cc:729] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575303e10edd446db43b7c363ecd132a, State: Initialized, Role: FOLLOWER
I20260806 19:34:46.112804 12848 consensus_queue.cc:260] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:46.113559 12848 ts_tablet_manager.cc:1428] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a: Time spent starting tablet: real 0.002s user 0.002s sys 0.002s
I20260806 19:34:46.253654 12866 raft_consensus.cc:482] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:34:46.254045 12866 raft_consensus.cc:504] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:46.255254 12866 leader_election.cc:288] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575303e10edd446db43b7c363ecd132a (127.10.25.2:34451), 529f95f853204b94b5f6e1b86a0bc092 (127.10.25.3:33861)
I20260806 19:34:46.256121 12690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "435873710cec40e18ef804ad781a7bd7" candidate_uuid: "3f0da279763745aa9289e16bc43ef630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "529f95f853204b94b5f6e1b86a0bc092" is_pre_election: true
I20260806 19:34:46.256119 12541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "435873710cec40e18ef804ad781a7bd7" candidate_uuid: "3f0da279763745aa9289e16bc43ef630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575303e10edd446db43b7c363ecd132a" is_pre_election: true
I20260806 19:34:46.256464 12690 raft_consensus.cc:2454] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0da279763745aa9289e16bc43ef630 in term 0.
I20260806 19:34:46.256511 12541 raft_consensus.cc:2454] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0da279763745aa9289e16bc43ef630 in term 0.
I20260806 19:34:46.257145 12318 leader_election.cc:302] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0da279763745aa9289e16bc43ef630, 529f95f853204b94b5f6e1b86a0bc092; no voters:
I20260806 19:34:46.257658 12866 raft_consensus.cc:2781] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:34:46.257843 12866 raft_consensus.cc:482] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:34:46.257984 12866 raft_consensus.cc:3037] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:46.260772 12866 raft_consensus.cc:504] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:46.261623 12866 leader_election.cc:288] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [CANDIDATE]: Term 1 election: Requested vote from peers 575303e10edd446db43b7c363ecd132a (127.10.25.2:34451), 529f95f853204b94b5f6e1b86a0bc092 (127.10.25.3:33861)
I20260806 19:34:46.262321 12690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "435873710cec40e18ef804ad781a7bd7" candidate_uuid: "3f0da279763745aa9289e16bc43ef630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "529f95f853204b94b5f6e1b86a0bc092"
I20260806 19:34:46.262356 12541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "435873710cec40e18ef804ad781a7bd7" candidate_uuid: "3f0da279763745aa9289e16bc43ef630" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575303e10edd446db43b7c363ecd132a"
I20260806 19:34:46.262603 12690 raft_consensus.cc:3037] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:46.262683 12541 raft_consensus.cc:3037] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:46.265333 12690 raft_consensus.cc:2454] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0da279763745aa9289e16bc43ef630 in term 1.
I20260806 19:34:46.266023 12318 leader_election.cc:302] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0da279763745aa9289e16bc43ef630, 529f95f853204b94b5f6e1b86a0bc092; no voters:
I20260806 19:34:46.266407 12866 raft_consensus.cc:2781] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:34:46.266469 12541 raft_consensus.cc:2454] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0da279763745aa9289e16bc43ef630 in term 1.
I20260806 19:34:46.266618 12866 raft_consensus.cc:686] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [term 1 LEADER]: Becoming Leader. State: Replica: 3f0da279763745aa9289e16bc43ef630, State: Running, Role: LEADER
I20260806 19:34:46.267215 12866 consensus_queue.cc:237] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } }
I20260806 19:34:46.271421 12224 catalog_manager.cc:5495] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f0da279763745aa9289e16bc43ef630 (127.10.25.1). New cstate: current_term: 1 leader_uuid: "3f0da279763745aa9289e16bc43ef630" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0da279763745aa9289e16bc43ef630" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 } health_report { overall_health: UNKNOWN } } }
I20260806 19:34:46.315346 12690 raft_consensus.cc:1261] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Refusing update from remote peer 3f0da279763745aa9289e16bc43ef630: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:34:46.315376 12537 raft_consensus.cc:1261] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Refusing update from remote peer 3f0da279763745aa9289e16bc43ef630: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:34:46.315995 12866 consensus_queue.cc:1035] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "529f95f853204b94b5f6e1b86a0bc092" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 33861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:34:46.316468 12855 consensus_queue.cc:1035] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [LEADER]: Connected to new peer: Peer: permanent_uuid: "575303e10edd446db43b7c363ecd132a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 34451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 19:34:46.341053 12820 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 19:34:47.992235 12224 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:34:47.992507 12224 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:34:47.999993 12224 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d1d41afcaaf4ee2bcd77d3913ba3a5d: Sending DeleteTablet for 3 replicas of tablet 435873710cec40e18ef804ad781a7bd7
I20260806 19:34:48.001591 12366 tablet_service.cc:1514] Processing DeleteTablet for tablet 435873710cec40e18ef804ad781a7bd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:47 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:36820
I20260806 19:34:48.001973 12665 tablet_service.cc:1514] Processing DeleteTablet for tablet 435873710cec40e18ef804ad781a7bd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:47 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:56290
I20260806 19:34:48.002095 12515 tablet_service.cc:1514] Processing DeleteTablet for tablet 435873710cec40e18ef804ad781a7bd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:34:47 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:42188
I20260806 19:34:48.003204 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 12299
I20260806 19:34:48.015329 12902 tablet_replica.cc:330] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092: stopping tablet replica
I20260806 19:34:48.015326 12901 tablet_replica.cc:330] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630: stopping tablet replica
I20260806 19:34:48.015640 12902 raft_consensus.cc:2229] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:34:48.015972 12902 raft_consensus.cc:2258] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:48.015959 12901 raft_consensus.cc:2229] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:34:48.016781 12901 raft_consensus.cc:2258] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:48.018014 12903 tablet_replica.cc:330] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a: stopping tablet replica
I20260806 19:34:48.018332 12903 raft_consensus.cc:2229] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:34:48.018385 12901 ts_tablet_manager.cc:1905] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:48.018363 12902 ts_tablet_manager.cc:1905] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:48.018784 12903 raft_consensus.cc:2258] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:34:48.020139 12903 ts_tablet_manager.cc:1905] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:34:48.026649 12902 ts_tablet_manager.cc:1918] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 19:34:48.026984 12902 log.cc:1196] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/435873710cec40e18ef804ad781a7bd7
I20260806 19:34:48.027297 12901 ts_tablet_manager.cc:1918] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 19:34:48.027521 12901 log.cc:1196] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/435873710cec40e18ef804ad781a7bd7
I20260806 19:34:48.027681 12902 ts_tablet_manager.cc:1939] T 435873710cec40e18ef804ad781a7bd7 P 529f95f853204b94b5f6e1b86a0bc092: Deleting consensus metadata
I20260806 19:34:48.028143 12901 ts_tablet_manager.cc:1939] T 435873710cec40e18ef804ad781a7bd7 P 3f0da279763745aa9289e16bc43ef630: Deleting consensus metadata
I20260806 19:34:48.029076 12903 ts_tablet_manager.cc:1918] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 19:34:48.029294 12903 log.cc:1196] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/435873710cec40e18ef804ad781a7bd7
I20260806 19:34:48.029330 12211 catalog_manager.cc:4841] TS 529f95f853204b94b5f6e1b86a0bc092 (127.10.25.3:33861): tablet 435873710cec40e18ef804ad781a7bd7 (table test-table [id=96f17f6094d644358d54e8aa000f4155]) successfully deleted
I20260806 19:34:48.029780 12211 catalog_manager.cc:4841] TS 3f0da279763745aa9289e16bc43ef630 (127.10.25.1:43133): tablet 435873710cec40e18ef804ad781a7bd7 (table test-table [id=96f17f6094d644358d54e8aa000f4155]) successfully deleted
I20260806 19:34:48.029841 12903 ts_tablet_manager.cc:1939] T 435873710cec40e18ef804ad781a7bd7 P 575303e10edd446db43b7c363ecd132a: Deleting consensus metadata
I20260806 19:34:48.031172 12210 catalog_manager.cc:4841] TS 575303e10edd446db43b7c363ecd132a (127.10.25.2:34451): tablet 435873710cec40e18ef804ad781a7bd7 (table test-table [id=96f17f6094d644358d54e8aa000f4155]) successfully deleted
Aug 06 19:34:48 dist-test-slave-ss4n krb5kdc[12175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044885, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:34:48.046058 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:48.005415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36834 (local address 127.10.25.1:43133)
0806 19:34:48.011052 (+ 5637us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:48.011064 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:48.019119 (+ 8055us) server_negotiation.cc:407] Connection header received
0806 19:34:48.019204 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:48.019210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:48.019360 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:48.019490 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:48.021839 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:48.023616 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:48.026012 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:48.026850 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:48.027020 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:48.038768 (+ 11748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:48.038791 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:48.038809 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:48.038857 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:48.042777 (+ 3920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:48.043657 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:48.043665 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:48.043667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:48.043836 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:48.044249 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:48.044252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:48.044254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:48.044673 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:48.044972 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:48.045272 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 19:34:48.045592 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5373,"thread_start_us":5115,"threads_started":1}
W20260806 19:34:48.047214 12435 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36834
W20260806 19:34:48.048216 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 12299: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:34:48.048599 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 12299
I20260806 19:34:48.076100 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 12445
Aug 06 19:34:48 dist-test-slave-ss4n krb5kdc[12175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044885, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:34:48.096941 12906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:48.077133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42190 (local address 127.10.25.2:34451)
0806 19:34:48.077750 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:48.077755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:48.077775 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:34:48.077838 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:48.077842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:48.077904 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:48.078012 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:48.079674 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:48.080653 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:48.082302 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:48.082870 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:48.083260 (+ 390us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:48.088865 (+ 5605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:48.088886 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:48.088904 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:48.088957 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:48.093081 (+ 4124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:48.094343 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:48.094357 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:48.094366 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:48.094574 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:48.095009 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:48.095016 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:48.095019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:48.095504 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:48.095900 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:48.096204 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 19:34:48.096520 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":370,"threads_started":1}
W20260806 19:34:48.097889 12584 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42190
W20260806 19:34:48.098840 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 12445: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:34:48.099069 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 12445
I20260806 19:34:48.122406 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 12594
Aug 06 19:34:48 dist-test-slave-ss4n krb5kdc[12175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044885, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:34:48.144129 12907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:48.123391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56316 (local address 127.10.25.3:33861)
0806 19:34:48.124000 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:48.124005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:48.124022 (+ 17us) server_negotiation.cc:407] Connection header received
0806 19:34:48.124083 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:48.124088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:48.124156 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:48.124270 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:48.125894 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:48.126760 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:48.128685 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:48.129297 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:48.129441 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:48.135656 (+ 6215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:48.135687 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:48.135716 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:48.135788 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:48.140044 (+ 4256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:48.141213 (+ 1169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:48.141225 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:48.141228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:48.141447 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:48.142029 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:48.142034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:48.142036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:48.142534 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:48.142854 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:48.143448 (+ 594us) server_negotiation.cc:299] Negotiation successful
0806 19:34:48.143723 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":344,"threads_started":1}
W20260806 19:34:48.144902 12730 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56316
W20260806 19:34:48.145855 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 12594: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:34:48.146102 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 12594
I20260806 19:34:48.176184 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 12191
I20260806 19:34:48.177614 12267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:34:48.311023 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 12191
2026-08-06T19:34:48Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9978 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:34:48 dist-test-slave-ss4n krb5kdc[12911](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:34:48 dist-test-slave-ss4n krb5kdc[12911](info): set up 2 sockets
Aug 06 19:34:48 dist-test-slave-ss4n krb5kdc[12911](info): commencing operation
krb5kdc: starting...
W20260806 19:34:49.889680 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.520s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:34:50 dist-test-slave-ss4n krb5kdc[12911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:34:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:34:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:34:50.328845 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:44613
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:44633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:44613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:50.955520 12925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:50.955868 12925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:50.956029 12925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:50.963716 12925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:34:50.963863 12925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:50.963948 12925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:50.964039 12925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:34:50.964102 12925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:34:50.974786 12925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:44633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:44613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:44613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:50.976300 12925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:50.977754 12925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:50.992347 12930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:50.992257 12933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:50.992282 12931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:50.992731 12925 server_base.cc:1034] running on GCE node
I20260806 19:34:50.994087 12925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:50.996191 12925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:50.997639 12925 hybrid_clock.cc:648] HybridClock initialized: now 1786044890997585 us; error 45 us; skew 500 ppm
I20260806 19:34:50.998080 12925 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:50 dist-test-slave-ss4n krb5kdc[12911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044890, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:51.001830 12925 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:34:51.003791 12925 webserver.cc:460] Webserver started at http://127.10.25.62:41055/ using document root <none> and password file <none>
I20260806 19:34:51.004585 12925 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:51.004706 12925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:51.005016 12925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:51.008767 12925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "e999ad3defb34faeaf21776fe5321366"
format_stamp: "Formatted at 2026-08-06 19:34:50 on dist-test-slave-ss4n"
server_key: "a1b4e53c6cfdd9a3823bd04c97cb53be"
server_key_iv: "8e5dd8bccfd8bf55de11e78374fc6262"
server_key_version: "encryptionkey@0"
I20260806 19:34:51.009826 12925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "e999ad3defb34faeaf21776fe5321366"
format_stamp: "Formatted at 2026-08-06 19:34:50 on dist-test-slave-ss4n"
server_key: "a1b4e53c6cfdd9a3823bd04c97cb53be"
server_key_iv: "8e5dd8bccfd8bf55de11e78374fc6262"
server_key_version: "encryptionkey@0"
I20260806 19:34:51.016623 12925 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:34:51.021066 12940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:51.022945 12925 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:34:51.023196 12925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "e999ad3defb34faeaf21776fe5321366"
format_stamp: "Formatted at 2026-08-06 19:34:50 on dist-test-slave-ss4n"
server_key: "a1b4e53c6cfdd9a3823bd04c97cb53be"
server_key_iv: "8e5dd8bccfd8bf55de11e78374fc6262"
server_key_version: "encryptionkey@0"
I20260806 19:34:51.023451 12925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:51.045837 12925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:51.051105 12925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:51.051433 12925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:51.074365 12925 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:44613
I20260806 19:34:51.074381 12991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:44613 every 8 connection(s)
I20260806 19:34:51.075964 12925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:34:51.080730 12992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:51.084965 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 12925
I20260806 19:34:51.085253 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:34:51.086138 10340 external_mini_cluster.cc:1428] Setting key 8b9ecf1646d7f389a811fa66bde17994
I20260806 19:34:51.096921 12992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366: Bootstrap starting.
I20260806 19:34:51.103060 12992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:51.104768 12992 log.cc:824] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366: Log is configured to *not* fsync() on all Append() calls
Aug 06 19:34:51 dist-test-slave-ss4n krb5kdc[12911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:51.108779 12992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366: No bootstrap required, opened a new log
I20260806 19:34:51.116819 12995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:51.088600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42448 (local address 127.10.25.62:44613)
0806 19:34:51.089629 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:51.089645 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:51.089694 (+ 49us) server_negotiation.cc:407] Connection header received
0806 19:34:51.091715 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:51.091744 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:51.092280 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:51.092699 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:51.095059 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:51.096689 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:51.099219 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:51.100232 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:51.100426 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:51.107257 (+ 6831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:51.107271 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:51.107298 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:51.107372 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:51.111470 (+ 4098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:51.112623 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:51.112654 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:51.112664 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:51.112988 (+ 324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:51.113476 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:51.113529 (+ 53us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:51.113533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:51.114182 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:51.114561 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:51.115050 (+ 489us) server_negotiation.cc:299] Negotiation successful
0806 19:34:51.115583 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":448,"threads_started":1}
I20260806 19:34:51.118919 12992 raft_consensus.cc:348] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e999ad3defb34faeaf21776fe5321366" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44613 } }
I20260806 19:34:51.119385 12992 raft_consensus.cc:374] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:51.119529 12992 raft_consensus.cc:729] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e999ad3defb34faeaf21776fe5321366, State: Initialized, Role: FOLLOWER
I20260806 19:34:51.120265 12992 consensus_queue.cc:260] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e999ad3defb34faeaf21776fe5321366" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44613 } }
I20260806 19:34:51.120621 12992 raft_consensus.cc:388] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:34:51.120808 12992 raft_consensus.cc:482] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:34:51.121009 12992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:51.123759 12992 raft_consensus.cc:504] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e999ad3defb34faeaf21776fe5321366" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44613 } }
I20260806 19:34:51.124429 12992 leader_election.cc:302] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e999ad3defb34faeaf21776fe5321366; no voters:
I20260806 19:34:51.125312 12992 leader_election.cc:288] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:34:51.125454 12997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:34:51.126973 12997 raft_consensus.cc:686] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [term 1 LEADER]: Becoming Leader. State: Replica: e999ad3defb34faeaf21776fe5321366, State: Running, Role: LEADER
I20260806 19:34:51.127749 12997 consensus_queue.cc:237] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e999ad3defb34faeaf21776fe5321366" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44613 } }
I20260806 19:34:51.128441 12992 sys_catalog.cc:564] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:34:51.131729 12998 sys_catalog.cc:455] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e999ad3defb34faeaf21776fe5321366" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e999ad3defb34faeaf21776fe5321366" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44613 } } }
I20260806 19:34:51.132462 12998 sys_catalog.cc:458] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [sys.catalog]: This master's current role is: LEADER
I20260806 19:34:51.132313 12999 sys_catalog.cc:455] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e999ad3defb34faeaf21776fe5321366. Latest consensus state: current_term: 1 leader_uuid: "e999ad3defb34faeaf21776fe5321366" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e999ad3defb34faeaf21776fe5321366" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44613 } } }
I20260806 19:34:51.132642 12999 sys_catalog.cc:458] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366 [sys.catalog]: This master's current role is: LEADER
I20260806 19:34:51.134192 13005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:34:51.144240 13005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:34:51.159116 13005 catalog_manager.cc:1347] Generated new cluster ID: 2f3ab1b701aa41369cb9d274828ae088
I20260806 19:34:51.159332 13005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:34:51.173452 13005 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:34:51.175573 13005 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:34:51.185238 13005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e999ad3defb34faeaf21776fe5321366: Generated new TSK 0
I20260806 19:34:51.186208 13005 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:34:51.262346 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44613
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:44633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:51.692368 13019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:51.692706 13019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:51.692867 13019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:51.700703 13019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:51.700839 13019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:51.701010 13019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:34:51.712567 13019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:44633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:51.714249 13019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:51.715801 13019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:51.731225 13027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:51.731664 13026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:51.731627 13029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:51.733888 13019 server_base.cc:1034] running on GCE node
I20260806 19:34:51.734771 13019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:51.736140 13019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:51.737771 13019 hybrid_clock.cc:648] HybridClock initialized: now 1786044891737671 us; error 83 us; skew 500 ppm
I20260806 19:34:51.738222 13019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:51 dist-test-slave-ss4n krb5kdc[12911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044891, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:51.742177 13019 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:34:51.744031 13019 webserver.cc:460] Webserver started at http://127.10.25.1:40193/ using document root <none> and password file <none>
I20260806 19:34:51.744894 13019 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:51.745057 13019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:51.745419 13019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:51.748984 13019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "45c9b977f98d4f3589f5f5e2e3a7437d"
format_stamp: "Formatted at 2026-08-06 19:34:51 on dist-test-slave-ss4n"
server_key: "d4db3e9f0c3f3952a9e57837097cf399"
server_key_iv: "f1e748351932efd825fc97830b5d8e8b"
server_key_version: "encryptionkey@0"
I20260806 19:34:51.750010 13019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "45c9b977f98d4f3589f5f5e2e3a7437d"
format_stamp: "Formatted at 2026-08-06 19:34:51 on dist-test-slave-ss4n"
server_key: "d4db3e9f0c3f3952a9e57837097cf399"
server_key_iv: "f1e748351932efd825fc97830b5d8e8b"
server_key_version: "encryptionkey@0"
I20260806 19:34:51.756449 13019 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260806 19:34:51.760593 13037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:51.762717 13019 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:34:51.762969 13019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "45c9b977f98d4f3589f5f5e2e3a7437d"
format_stamp: "Formatted at 2026-08-06 19:34:51 on dist-test-slave-ss4n"
server_key: "d4db3e9f0c3f3952a9e57837097cf399"
server_key_iv: "f1e748351932efd825fc97830b5d8e8b"
server_key_version: "encryptionkey@0"
I20260806 19:34:51.763238 13019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:51.780481 13019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:51.786741 13019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:51.787329 13019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:51.788614 13019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:34:51.790318 13019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:51.790450 13019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:51.790620 13019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:51.790699 13019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:51.835471 13019 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:43131
I20260806 19:34:51.835511 13152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:43131 every 8 connection(s)
I20260806 19:34:51.836877 13019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:34:51.843510 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 13019
I20260806 19:34:51.843801 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:34:51.844491 10340 external_mini_cluster.cc:1428] Setting key fef114b52615137883cf521d2356d9b3
Aug 06 19:34:51 dist-test-slave-ss4n krb5kdc[12911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044891, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
I20260806 19:34:51.870055 13155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:51.840948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:43159 (local address 127.10.25.62:44613)
0806 19:34:51.841750 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:51.841759 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:51.843219 (+ 1460us) server_negotiation.cc:407] Connection header received
0806 19:34:51.846826 (+ 3607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:51.846834 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:51.846929 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:51.847031 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:51.849897 (+ 2866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:51.851126 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:51.853037 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:51.853905 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:51.854095 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:51.860260 (+ 6165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:51.860268 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:51.860271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:51.860321 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:51.865464 (+ 5143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:51.866700 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:51.866711 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:51.866717 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:51.866928 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:51.867610 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:51.867618 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:51.867624 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:51.868125 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:51.868318 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:51.869299 (+ 981us) server_negotiation.cc:299] Negotiation successful
0806 19:34:51.869638 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":397,"threads_started":1}
I20260806 19:34:51.871299 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:51.841801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44613 (local address 127.10.25.1:43159)
0806 19:34:51.842926 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:51.842997 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:51.846407 (+ 3410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:51.847355 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:51.847365 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:51.848482 (+ 1117us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:51.849630 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:51.849668 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:51.851406 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:51.851410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:51.852833 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:51.852850 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:51.854244 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:51.854250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:51.854825 (+ 575us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:51.855965 (+ 1140us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:51.856010 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:51.859656 (+ 3646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:51.865766 (+ 6110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:51.865798 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:51.865816 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:51.866492 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:51.867119 (+ 627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:51.867124 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:51.867127 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:51.867448 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:51.868319 (+ 871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:51.868329 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:51.868814 (+ 485us) client_negotiation.cc:769] Sending connection context
0806 19:34:51.869182 (+ 368us) client_negotiation.cc:240] Negotiation successful
0806 19:34:51.869847 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":402,"threads_started":1}
I20260806 19:34:51.873346 13153 heartbeater.cc:347] Connected to a master server at 127.10.25.62:44613
I20260806 19:34:51.873781 13153 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
I20260806 19:34:51.874662 13153 heartbeater.cc:511] Master 127.10.25.62:44613 requested a full tablet report, sending...
I20260806 19:34:51.877511 12957 ts_manager.cc:194] Registered new tserver with Master: 45c9b977f98d4f3589f5f5e2e3a7437d (127.10.25.1:43131)
I20260806 19:34:51.880012 12957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:43159
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:34:51.909366 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44613
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:44633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:52.312750 13159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:52.313102 13159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:52.313211 13159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:52.321039 13159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:52.321183 13159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:52.321305 13159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:34:52.334053 13159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:44633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:52.335609 13159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:52.337133 13159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:52.352782 13164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:52.352782 13165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:52.352887 13167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:52.354846 13159 server_base.cc:1034] running on GCE node
I20260806 19:34:52.355731 13159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:52.356930 13159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:52.358208 13159 hybrid_clock.cc:648] HybridClock initialized: now 1786044892358129 us; error 75 us; skew 500 ppm
I20260806 19:34:52.358616 13159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:52 dist-test-slave-ss4n krb5kdc[12911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044892, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:52.362460 13159 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:34:52.364408 13159 webserver.cc:460] Webserver started at http://127.10.25.2:35265/ using document root <none> and password file <none>
I20260806 19:34:52.365227 13159 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:52.365389 13159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:52.365803 13159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:52.369455 13159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "172f3973ac5e4b039ff146d2b03bcbc0"
format_stamp: "Formatted at 2026-08-06 19:34:52 on dist-test-slave-ss4n"
server_key: "46a5cecc17a7017d13bc88fb32948f85"
server_key_iv: "28b5d00578ffb6c262e0cf684079c5ca"
server_key_version: "encryptionkey@0"
I20260806 19:34:52.370496 13159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "172f3973ac5e4b039ff146d2b03bcbc0"
format_stamp: "Formatted at 2026-08-06 19:34:52 on dist-test-slave-ss4n"
server_key: "46a5cecc17a7017d13bc88fb32948f85"
server_key_iv: "28b5d00578ffb6c262e0cf684079c5ca"
server_key_version: "encryptionkey@0"
I20260806 19:34:52.378247 13159 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.004s
I20260806 19:34:52.382512 13176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:52.384222 13159 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:34:52.384466 13159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "172f3973ac5e4b039ff146d2b03bcbc0"
format_stamp: "Formatted at 2026-08-06 19:34:52 on dist-test-slave-ss4n"
server_key: "46a5cecc17a7017d13bc88fb32948f85"
server_key_iv: "28b5d00578ffb6c262e0cf684079c5ca"
server_key_version: "encryptionkey@0"
I20260806 19:34:52.384704 13159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:52.403668 13159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:52.409228 13159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:52.409621 13159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:52.410871 13159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:34:52.412493 13159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:52.412581 13159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:52.412739 13159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:52.412798 13159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:52.455703 13159 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:46861
I20260806 19:34:52.455747 13294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:46861 every 8 connection(s)
I20260806 19:34:52.457209 13159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:34:52.459599 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 13159
I20260806 19:34:52.459846 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:34:52.460342 10340 external_mini_cluster.cc:1428] Setting key 6c8fe4e63d8d2b573996a2d118bea5af
Aug 06 19:34:52 dist-test-slave-ss4n krb5kdc[12911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044892, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
I20260806 19:34:52.487423 13298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:52.460728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:58099 (local address 127.10.25.62:44613)
0806 19:34:52.461322 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:52.461327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:52.462765 (+ 1438us) server_negotiation.cc:407] Connection header received
0806 19:34:52.464974 (+ 2209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:52.464981 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:52.465092 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:52.465234 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:52.467113 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:52.468515 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:52.471323 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:52.472254 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:52.472466 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:52.478355 (+ 5889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:52.478367 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:52.478376 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:52.478438 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:52.482956 (+ 4518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:52.484129 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:52.484140 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:52.484146 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:52.484359 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:52.484978 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:52.484983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:52.484987 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:52.485432 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:52.485655 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:52.486778 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0806 19:34:52.487103 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":304,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
I20260806 19:34:52.488900 13299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:52.461465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44613 (local address 127.10.25.2:58099)
0806 19:34:52.462565 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:52.462619 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:52.464616 (+ 1997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:52.465474 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:52.465540 (+ 66us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:52.466204 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:52.466888 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:52.466909 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:52.468775 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:52.468784 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:52.470898 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:52.471032 (+ 134us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:52.472624 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:52.472631 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:52.472903 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:52.474268 (+ 1365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:52.474311 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:52.477929 (+ 3618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:52.483248 (+ 5319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:52.483259 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:52.483278 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:52.483928 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:52.484540 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:52.484545 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:52.484547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:52.484825 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:52.485636 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:52.485643 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:52.486163 (+ 520us) client_negotiation.cc:769] Sending connection context
0806 19:34:52.486605 (+ 442us) client_negotiation.cc:240] Negotiation successful
0806 19:34:52.487322 (+ 717us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":409,"threads_started":1}
I20260806 19:34:52.491321 13295 heartbeater.cc:347] Connected to a master server at 127.10.25.62:44613
I20260806 19:34:52.491865 13295 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:52.493039 13295 heartbeater.cc:511] Master 127.10.25.62:44613 requested a full tablet report, sending...
I20260806 19:34:52.495630 12957 ts_manager.cc:194] Registered new tserver with Master: 172f3973ac5e4b039ff146d2b03bcbc0 (127.10.25.2:46861)
I20260806 19:34:52.496555 12957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:58099
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:34:52.521663 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44613
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:44633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:34:52.884760 13153 heartbeater.cc:503] Master 127.10.25.62:44613 was elected leader, sending a full tablet report...
W20260806 19:34:52.925861 13302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:52.926219 13302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:52.926381 13302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:52.934747 13302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:52.934901 13302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:52.935032 13302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:34:52.945683 13302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:44633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:52.947201 13302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:52.948657 13302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:52.965272 13307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:52.965215 13310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:52.965339 13308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:52.966948 13302 server_base.cc:1034] running on GCE node
I20260806 19:34:52.967799 13302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:52.969019 13302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:52.970306 13302 hybrid_clock.cc:648] HybridClock initialized: now 1786044892970238 us; error 70 us; skew 500 ppm
I20260806 19:34:52.971067 13302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:52 dist-test-slave-ss4n krb5kdc[12911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044892, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:52.975075 13302 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:34:52.976801 13302 webserver.cc:460] Webserver started at http://127.10.25.3:43263/ using document root <none> and password file <none>
I20260806 19:34:52.977697 13302 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:52.977814 13302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:52.978319 13302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:52.982020 13302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "d78a782c9c7a4c878922266b717a2897"
format_stamp: "Formatted at 2026-08-06 19:34:52 on dist-test-slave-ss4n"
server_key: "d6949150d02d5c9b732a605180e05bb6"
server_key_iv: "5595587cc7ee1cbf1a2404b7668b9f08"
server_key_version: "encryptionkey@0"
I20260806 19:34:52.983057 13302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "d78a782c9c7a4c878922266b717a2897"
format_stamp: "Formatted at 2026-08-06 19:34:52 on dist-test-slave-ss4n"
server_key: "d6949150d02d5c9b732a605180e05bb6"
server_key_iv: "5595587cc7ee1cbf1a2404b7668b9f08"
server_key_version: "encryptionkey@0"
I20260806 19:34:52.989740 13302 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 19:34:52.994666 13318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:52.996373 13302 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 19:34:52.996567 13302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "d78a782c9c7a4c878922266b717a2897"
format_stamp: "Formatted at 2026-08-06 19:34:52 on dist-test-slave-ss4n"
server_key: "d6949150d02d5c9b732a605180e05bb6"
server_key_iv: "5595587cc7ee1cbf1a2404b7668b9f08"
server_key_version: "encryptionkey@0"
I20260806 19:34:52.996770 13302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:53.044435 13302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:53.049654 13302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:53.049945 13302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:53.050989 13302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:34:53.052449 13302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:53.052525 13302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:53.052614 13302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:53.052668 13302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:53.094892 13302 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:34267
I20260806 19:34:53.095012 13437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:34267 every 8 connection(s)
I20260806 19:34:53.096289 13302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:34:53.100312 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 13302
I20260806 19:34:53.100539 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:34:53.101269 10340 external_mini_cluster.cc:1428] Setting key fcbebb7afa0776b159004a7baaca719c
Aug 06 19:34:53 dist-test-slave-ss4n krb5kdc[12911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044892, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:53.123683 13440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:53.099240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:45645 (local address 127.10.25.62:44613)
0806 19:34:53.099754 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:53.099759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:53.100914 (+ 1155us) server_negotiation.cc:407] Connection header received
0806 19:34:53.104044 (+ 3130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:53.104050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:53.104180 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:53.104284 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:53.106549 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:53.107371 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:53.109233 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:53.109843 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:53.109977 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:53.114436 (+ 4459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:53.114444 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:53.114448 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:53.114508 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:53.118496 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:53.120001 (+ 1505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:53.120006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:53.120009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:53.120220 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:53.120979 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:53.120983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:53.120985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:53.121328 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:53.121618 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:53.122972 (+ 1354us) server_negotiation.cc:299] Negotiation successful
0806 19:34:53.123305 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":284,"threads_started":1}
I20260806 19:34:53.124708 13441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:53.099669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44613 (local address 127.10.25.3:45645)
0806 19:34:53.100598 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:53.100670 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:53.103703 (+ 3033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:53.104605 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:53.104618 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:53.105662 (+ 1044us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:53.106337 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:53.106369 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:53.107584 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:53.107589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:53.109037 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:53.109057 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:53.110110 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:53.110117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:53.110368 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:53.111200 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:53.111226 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:53.114010 (+ 2784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:53.118989 (+ 4979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:53.119001 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:53.119025 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:53.119775 (+ 750us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:53.120437 (+ 662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:53.120443 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:53.120447 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:53.120812 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:53.121607 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:53.121616 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:53.122116 (+ 500us) client_negotiation.cc:769] Sending connection context
0806 19:34:53.122710 (+ 594us) client_negotiation.cc:240] Negotiation successful
0806 19:34:53.123271 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":410,"threads_started":1}
I20260806 19:34:53.126868 13438 heartbeater.cc:347] Connected to a master server at 127.10.25.62:44613
I20260806 19:34:53.127184 13438 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:53.128121 13438 heartbeater.cc:511] Master 127.10.25.62:44613 requested a full tablet report, sending...
I20260806 19:34:53.130419 12957 ts_manager.cc:194] Registered new tserver with Master: d78a782c9c7a4c878922266b717a2897 (127.10.25.3:34267)
I20260806 19:34:53.132117 12957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:45645
I20260806 19:34:53.133126 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 19:34:53.174122 13440 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:34:53.170959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43564 (local address 127.10.25.62:44613)
0806 19:34:53.171239 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:53.171249 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:53.171689 (+ 440us) server_negotiation.cc:407] Connection header received
0806 19:34:53.171925 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:53.171933 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:53.172043 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:53.172170 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:53.173748 (+ 1578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43564: BlockingRecv error: recv got EOF from 127.0.0.1:43564 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 19:34:53.177788 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 13019
W20260806 19:34:53.182372 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 13019: Not authorized: Client connection negotiation failed: client connection to 127.10.25.1:43131: server requires authentication, but client does not have Kerberos credentials available
I20260806 19:34:53.182629 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 13019
I20260806 19:34:53.202760 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 13159
W20260806 19:34:53.205866 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 13159: Not authorized: Client connection negotiation failed: client connection to 127.10.25.2:46861: server requires authentication, but client does not have Kerberos credentials available
I20260806 19:34:53.206104 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 13159
I20260806 19:34:53.225358 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 13302
W20260806 19:34:53.228639 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 13302: Not authorized: Client connection negotiation failed: client connection to 127.10.25.3:34267: server requires authentication, but client does not have Kerberos credentials available
I20260806 19:34:53.228819 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 13302
I20260806 19:34:53.247377 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 12925
I20260806 19:34:53.248973 12987 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:34:53.365170 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 12925
2026-08-06T19:34:53Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (5043 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:34:53 dist-test-slave-ss4n krb5kdc[13456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:34:53 dist-test-slave-ss4n krb5kdc[13456](info): set up 2 sockets
Aug 06 19:34:53 dist-test-slave-ss4n krb5kdc[13456](info): commencing operation
krb5kdc: starting...
W20260806 19:34:54.573760 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:34:54 dist-test-slave-ss4n krb5kdc[13456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:34:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:34:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:34:54.834757 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:36777
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:37943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:36777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:55.205703 13469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:55.206058 13469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:55.206156 13469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:55.214227 13469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:34:55.214397 13469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:55.214453 13469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:55.214521 13469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:34:55.214583 13469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:34:55.226717 13469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:37943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:36777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:36777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:55.228170 13469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:55.229791 13469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:55.243891 13474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:55.243935 13477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:55.243960 13475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:55.245349 13469 server_base.cc:1034] running on GCE node
I20260806 19:34:55.246483 13469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:55.248270 13469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:55.249719 13469 hybrid_clock.cc:648] HybridClock initialized: now 1786044895249610 us; error 75 us; skew 500 ppm
I20260806 19:34:55.250447 13469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:55 dist-test-slave-ss4n krb5kdc[13456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044895, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:55.254818 13469 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:34:55.256940 13469 webserver.cc:460] Webserver started at http://127.10.25.62:46373/ using document root <none> and password file <none>
I20260806 19:34:55.258033 13469 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:55.258208 13469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:55.258608 13469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:55.262463 13469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "3ba0e1e4328a41a8a5ed5a8e8ec5cbdb"
format_stamp: "Formatted at 2026-08-06 19:34:55 on dist-test-slave-ss4n"
server_key: "8525cadccc4a16f067b476c80a7380c3"
server_key_iv: "16164365983406c00d93821914b840d1"
server_key_version: "encryptionkey@0"
I20260806 19:34:55.263507 13469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "3ba0e1e4328a41a8a5ed5a8e8ec5cbdb"
format_stamp: "Formatted at 2026-08-06 19:34:55 on dist-test-slave-ss4n"
server_key: "8525cadccc4a16f067b476c80a7380c3"
server_key_iv: "16164365983406c00d93821914b840d1"
server_key_version: "encryptionkey@0"
I20260806 19:34:55.271458 13469 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260806 19:34:55.276592 13486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:55.278568 13469 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:34:55.278877 13469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "3ba0e1e4328a41a8a5ed5a8e8ec5cbdb"
format_stamp: "Formatted at 2026-08-06 19:34:55 on dist-test-slave-ss4n"
server_key: "8525cadccc4a16f067b476c80a7380c3"
server_key_iv: "16164365983406c00d93821914b840d1"
server_key_version: "encryptionkey@0"
I20260806 19:34:55.279158 13469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:55.298497 13469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:55.305182 13469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:55.305636 13469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:55.329002 13469 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:36777
I20260806 19:34:55.329064 13539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:36777 every 8 connection(s)
I20260806 19:34:55.330753 13469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:34:55.331650 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 13469
I20260806 19:34:55.331993 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:34:55.333034 10340 external_mini_cluster.cc:1428] Setting key af0fe0f6e6603cda4d9e5ce22059aae9
I20260806 19:34:55.336239 13540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 19:34:55 dist-test-slave-ss4n krb5kdc[13456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:55.349948 13540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb: Bootstrap starting.
I20260806 19:34:55.354147 13540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:55.355437 13540 log.cc:824] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb: Log is configured to *not* fsync() on all Append() calls
I20260806 19:34:55.360181 13540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb: No bootstrap required, opened a new log
I20260806 19:34:55.360816 13543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:55.335513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51004 (local address 127.10.25.62:36777)
0806 19:34:55.336759 (+ 1246us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:55.336773 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:55.336832 (+ 59us) server_negotiation.cc:407] Connection header received
0806 19:34:55.338825 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:55.338853 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:55.339357 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:55.339927 (+ 570us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:55.341951 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:55.343402 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:55.344777 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:55.345670 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:55.345935 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:55.351112 (+ 5177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:55.351122 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:55.351140 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:55.351187 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:55.355211 (+ 4024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:55.356340 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:55.356349 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:55.356357 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:55.356685 (+ 328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:55.357164 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:55.357170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:55.357179 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:55.357965 (+ 786us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:55.358333 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:55.358915 (+ 582us) server_negotiation.cc:299] Negotiation successful
0806 19:34:55.359440 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":880,"thread_start_us":532,"threads_started":1}
I20260806 19:34:55.366544 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:34:55.370831 13540 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba0e1e4328a41a8a5ed5a8e8ec5cbdb" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36777 } }
I20260806 19:34:55.371363 13540 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:55.371492 13540 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb, State: Initialized, Role: FOLLOWER
I20260806 19:34:55.372275 13540 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba0e1e4328a41a8a5ed5a8e8ec5cbdb" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36777 } }
I20260806 19:34:55.372579 13540 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:34:55.372751 13540 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:34:55.372991 13540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:55.376376 13540 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba0e1e4328a41a8a5ed5a8e8ec5cbdb" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36777 } }
I20260806 19:34:55.377189 13540 leader_election.cc:302] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb; no voters:
I20260806 19:34:55.378072 13540 leader_election.cc:288] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:34:55.378198 13546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:34:55.381404 13540 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:34:55.385730 13546 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [term 1 LEADER]: Becoming Leader. State: Replica: 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb, State: Running, Role: LEADER
I20260806 19:34:55.386689 13546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba0e1e4328a41a8a5ed5a8e8ec5cbdb" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36777 } }
I20260806 19:34:55.390750 13548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb. Latest consensus state: current_term: 1 leader_uuid: "3ba0e1e4328a41a8a5ed5a8e8ec5cbdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba0e1e4328a41a8a5ed5a8e8ec5cbdb" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36777 } } }
I20260806 19:34:55.391346 13548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [sys.catalog]: This master's current role is: LEADER
I20260806 19:34:55.392755 13547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ba0e1e4328a41a8a5ed5a8e8ec5cbdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba0e1e4328a41a8a5ed5a8e8ec5cbdb" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36777 } } }
I20260806 19:34:55.393126 13547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb [sys.catalog]: This master's current role is: LEADER
I20260806 19:34:55.396981 13558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:34:55.404938 13558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:34:55.419569 13566 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 19:34:55.424489 13558 catalog_manager.cc:1347] Generated new cluster ID: cc93a8cfa3464b5e88d2d635089b6de1
I20260806 19:34:55.424669 13558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:34:55.439088 13543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:55.427453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51018 (local address 127.10.25.62:36777)
0806 19:34:55.427643 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:55.427649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:55.428367 (+ 718us) server_negotiation.cc:407] Connection header received
0806 19:34:55.428559 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:55.428565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:55.428655 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:55.428779 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:55.430228 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:55.431431 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:55.433597 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:55.434290 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:55.434444 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:55.435916 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:55.435926 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:55.435932 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:34:55.437649 (+ 1717us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:55.437756 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:34:55.437810 (+ 54us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:55.438030 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:55.438268 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 19:34:55.438450 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 19:34:55.478225 13558 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:34:55.479892 13558 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:34:55.489907 13558 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ba0e1e4328a41a8a5ed5a8e8ec5cbdb: Generated new TSK 0
I20260806 19:34:55.491245 13558 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:34:55.513063 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 13469
I20260806 19:34:55.514654 13535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:34:55.619849 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 13469
2026-08-06T19:34:55Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2249 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:34:55 dist-test-slave-ss4n krb5kdc[13576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:34:55 dist-test-slave-ss4n krb5kdc[13576](info): set up 2 sockets
Aug 06 19:34:55 dist-test-slave-ss4n krb5kdc[13576](info): commencing operation
krb5kdc: starting...
W20260806 19:34:56.827401 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.005s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:34:56 dist-test-slave-ss4n krb5kdc[13576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:34:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:34:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:34:57.081907 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:33227
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:34941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:33227
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:57.448860 13589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:57.449275 13589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:57.449450 13589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:57.457142 13589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:34:57.457340 13589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:57.457449 13589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:57.457578 13589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:34:57.457656 13589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:34:57.468451 13589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:34941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:33227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:33227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:57.470499 13589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:57.472128 13589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:57.488196 13595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:57.488299 13597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:57.488315 13594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:57.488783 13589 server_base.cc:1034] running on GCE node
I20260806 19:34:57.489849 13589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:57.491631 13589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:57.492910 13589 hybrid_clock.cc:648] HybridClock initialized: now 1786044897492854 us; error 58 us; skew 500 ppm
I20260806 19:34:57.493332 13589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:57 dist-test-slave-ss4n krb5kdc[13576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044897, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:57.497301 13589 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:34:57.499183 13589 webserver.cc:460] Webserver started at http://127.10.25.62:38323/ using document root <none> and password file <none>
I20260806 19:34:57.499958 13589 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:57.500113 13589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:57.500454 13589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:57.504160 13589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "64a4229c9bcf4e1f93912a19f80f6404"
format_stamp: "Formatted at 2026-08-06 19:34:57 on dist-test-slave-ss4n"
server_key: "7f40ab60366bbb8a0dff36b173c3e744"
server_key_iv: "23bb069ca0f6136b4b72ee467728818e"
server_key_version: "encryptionkey@0"
I20260806 19:34:57.505098 13589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "64a4229c9bcf4e1f93912a19f80f6404"
format_stamp: "Formatted at 2026-08-06 19:34:57 on dist-test-slave-ss4n"
server_key: "7f40ab60366bbb8a0dff36b173c3e744"
server_key_iv: "23bb069ca0f6136b4b72ee467728818e"
server_key_version: "encryptionkey@0"
I20260806 19:34:57.511833 13589 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 19:34:57.516216 13604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:57.518571 13589 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:34:57.518843 13589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "64a4229c9bcf4e1f93912a19f80f6404"
format_stamp: "Formatted at 2026-08-06 19:34:57 on dist-test-slave-ss4n"
server_key: "7f40ab60366bbb8a0dff36b173c3e744"
server_key_iv: "23bb069ca0f6136b4b72ee467728818e"
server_key_version: "encryptionkey@0"
I20260806 19:34:57.519102 13589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:57.552989 13589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:57.559248 13589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:57.559810 13589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:57.583197 13589 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:33227
I20260806 19:34:57.583246 13658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:33227 every 8 connection(s)
I20260806 19:34:57.584738 13589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:34:57.589262 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 13589
I20260806 19:34:57.589465 13659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:34:57.589571 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:34:57.590538 10340 external_mini_cluster.cc:1428] Setting key 556a814a1c4191a027d51c9b59e9cd6e
I20260806 19:34:57.606457 13659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404: Bootstrap starting.
Aug 06 19:34:57 dist-test-slave-ss4n krb5kdc[13576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:57.610363 13659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404: Neither blocks nor log segments found. Creating new log.
I20260806 19:34:57.611781 13659 log.cc:824] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404: Log is configured to *not* fsync() on all Append() calls
I20260806 19:34:57.615263 13659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404: No bootstrap required, opened a new log
I20260806 19:34:57.618866 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:57.592946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44222 (local address 127.10.25.62:33227)
0806 19:34:57.593922 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:57.593934 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:57.593982 (+ 48us) server_negotiation.cc:407] Connection header received
0806 19:34:57.595589 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:57.595615 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:57.596444 (+ 829us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:57.597037 (+ 593us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:57.599118 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:57.600667 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:57.602685 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:57.603620 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:57.603796 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:57.609294 (+ 5498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:57.609305 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:57.609323 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:57.609379 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:57.613828 (+ 4449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:57.614783 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:57.614791 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:57.614798 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:57.615003 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:57.615469 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:57.615474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:57.615476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:57.616117 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:57.616531 (+ 414us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:57.617013 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 19:34:57.617609 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":486,"threads_started":1}
I20260806 19:34:57.624434 13659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a4229c9bcf4e1f93912a19f80f6404" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 33227 } }
I20260806 19:34:57.624953 13659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:34:57.625097 13659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64a4229c9bcf4e1f93912a19f80f6404, State: Initialized, Role: FOLLOWER
I20260806 19:34:57.625799 13659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a4229c9bcf4e1f93912a19f80f6404" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 33227 } }
I20260806 19:34:57.626068 13659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:34:57.626227 13659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:34:57.626415 13659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:34:57.628960 13659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a4229c9bcf4e1f93912a19f80f6404" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 33227 } }
I20260806 19:34:57.629721 13659 leader_election.cc:302] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64a4229c9bcf4e1f93912a19f80f6404; no voters:
I20260806 19:34:57.630568 13659 leader_election.cc:288] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:34:57.630767 13665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:34:57.632827 13665 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [term 1 LEADER]: Becoming Leader. State: Replica: 64a4229c9bcf4e1f93912a19f80f6404, State: Running, Role: LEADER
I20260806 19:34:57.633406 13659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:34:57.633986 13665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a4229c9bcf4e1f93912a19f80f6404" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 33227 } }
I20260806 19:34:57.637884 13666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64a4229c9bcf4e1f93912a19f80f6404" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a4229c9bcf4e1f93912a19f80f6404" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 33227 } } }
I20260806 19:34:57.637854 13667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64a4229c9bcf4e1f93912a19f80f6404. Latest consensus state: current_term: 1 leader_uuid: "64a4229c9bcf4e1f93912a19f80f6404" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a4229c9bcf4e1f93912a19f80f6404" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 33227 } } }
I20260806 19:34:57.638348 13666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [sys.catalog]: This master's current role is: LEADER
I20260806 19:34:57.638386 13667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404 [sys.catalog]: This master's current role is: LEADER
I20260806 19:34:57.639833 13673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:34:57.646704 13673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:34:57.657655 13673 catalog_manager.cc:1347] Generated new cluster ID: dbb4a896dab04a97978c086a96ca8c0b
I20260806 19:34:57.657816 13673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:34:57.668360 13673 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:34:57.670568 13673 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:34:57.679410 13673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64a4229c9bcf4e1f93912a19f80f6404: Generated new TSK 0
I20260806 19:34:57.680377 13673 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:34:57.749092 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:33227
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:34941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:58.165760 13687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:58.166085 13687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:58.166240 13687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:58.174041 13687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:58.174192 13687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:58.174388 13687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:34:58.184883 13687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:34941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:33227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:58.186417 13687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:58.187798 13687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:58.206617 13695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:58.206722 13693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:58.206653 13692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:58.207556 13687 server_base.cc:1034] running on GCE node
I20260806 19:34:58.208370 13687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:58.209733 13687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:58.210996 13687 hybrid_clock.cc:648] HybridClock initialized: now 1786044898210938 us; error 47 us; skew 500 ppm
I20260806 19:34:58.211423 13687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:58 dist-test-slave-ss4n krb5kdc[13576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044898, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:58.220238 13687 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:34:58.222306 13687 webserver.cc:460] Webserver started at http://127.10.25.1:33487/ using document root <none> and password file <none>
I20260806 19:34:58.223168 13687 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:58.223330 13687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:58.223694 13687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:58.227108 13687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "cbfd843b0cc34ba78d2d2188eaf21254"
format_stamp: "Formatted at 2026-08-06 19:34:58 on dist-test-slave-ss4n"
server_key: "b2ea5c8feb0f51df217f6af8457f6c41"
server_key_iv: "bc87f856774cca779043e212de3392c5"
server_key_version: "encryptionkey@0"
I20260806 19:34:58.228026 13687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "cbfd843b0cc34ba78d2d2188eaf21254"
format_stamp: "Formatted at 2026-08-06 19:34:58 on dist-test-slave-ss4n"
server_key: "b2ea5c8feb0f51df217f6af8457f6c41"
server_key_iv: "bc87f856774cca779043e212de3392c5"
server_key_version: "encryptionkey@0"
I20260806 19:34:58.234473 13687 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260806 19:34:58.239523 13702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:58.241691 13687 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.002s
I20260806 19:34:58.241955 13687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "cbfd843b0cc34ba78d2d2188eaf21254"
format_stamp: "Formatted at 2026-08-06 19:34:58 on dist-test-slave-ss4n"
server_key: "b2ea5c8feb0f51df217f6af8457f6c41"
server_key_iv: "bc87f856774cca779043e212de3392c5"
server_key_version: "encryptionkey@0"
I20260806 19:34:58.242242 13687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:58.279632 13687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:58.285099 13687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:58.285473 13687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:58.286698 13687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:34:58.288403 13687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:58.288525 13687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:58.288702 13687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:58.288784 13687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:58.330765 13687 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:46487
I20260806 19:34:58.330806 13818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:46487 every 8 connection(s)
I20260806 19:34:58.332229 13687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:34:58.342319 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 13687
I20260806 19:34:58.342542 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:34:58.343294 10340 external_mini_cluster.cc:1428] Setting key 98c076a5c1257bf50b5540d26f55466b
Aug 06 19:34:58 dist-test-slave-ss4n krb5kdc[13576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044898, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:58.360900 13821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:58.335351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:46269 (local address 127.10.25.62:33227)
0806 19:34:58.335956 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:58.335961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:58.336835 (+ 874us) server_negotiation.cc:407] Connection header received
0806 19:34:58.339063 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:58.339070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:58.339190 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:58.339347 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:58.341440 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:58.342657 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:58.344946 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:58.345610 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:58.345727 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:58.351308 (+ 5581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:58.351320 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:58.351329 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:58.351410 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:58.355976 (+ 4566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:58.357631 (+ 1655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:58.357638 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:58.357642 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:58.357858 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:58.358555 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:58.358560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:58.358562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:58.358978 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:58.359135 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:58.360251 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0806 19:34:58.360548 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":334,"threads_started":1}
I20260806 19:34:58.362895 13822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:58.335819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:33227 (local address 127.10.25.1:46269)
0806 19:34:58.336638 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:58.336691 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:58.338688 (+ 1997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:58.339672 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:58.339682 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:58.340308 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:58.341142 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:58.341165 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:58.343073 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:58.343078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:58.344770 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:58.344787 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:58.345941 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:58.345952 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:58.346396 (+ 444us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:58.347375 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:58.347427 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:58.350811 (+ 3384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:58.356298 (+ 5487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:58.356314 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:58.356339 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:58.357313 (+ 974us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:58.358022 (+ 709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:58.358031 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:58.358048 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:58.358373 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:58.359157 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:58.359170 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:58.359751 (+ 581us) client_negotiation.cc:769] Sending connection context
0806 19:34:58.360142 (+ 391us) client_negotiation.cc:240] Negotiation successful
0806 19:34:58.360894 (+ 752us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":321,"threads_started":1}
I20260806 19:34:58.365159 13819 heartbeater.cc:347] Connected to a master server at 127.10.25.62:33227
I20260806 19:34:58.365599 13819 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:58.366571 13819 heartbeater.cc:511] Master 127.10.25.62:33227 requested a full tablet report, sending...
I20260806 19:34:58.369213 13623 ts_manager.cc:194] Registered new tserver with Master: cbfd843b0cc34ba78d2d2188eaf21254 (127.10.25.1:46487)
I20260806 19:34:58.370895 13623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:46269
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:34:58.407554 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:33227
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:34941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:34:58.797711 13825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:58.798095 13825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:58.798276 13825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:58.806137 13825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:58.806324 13825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:58.806519 13825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:34:58.817940 13825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:34941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:33227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:58.819583 13825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:58.821149 13825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:58.837982 13830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:58.838107 13831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:58.838152 13833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:58.839416 13825 server_base.cc:1034] running on GCE node
I20260806 19:34:58.840265 13825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:58.841383 13825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:58.842656 13825 hybrid_clock.cc:648] HybridClock initialized: now 1786044898842592 us; error 51 us; skew 500 ppm
I20260806 19:34:58.842994 13825 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:58 dist-test-slave-ss4n krb5kdc[13576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044898, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:58.846777 13825 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:34:58.849164 13825 webserver.cc:460] Webserver started at http://127.10.25.2:41027/ using document root <none> and password file <none>
I20260806 19:34:58.850131 13825 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:58.850270 13825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:58.850646 13825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:58.854437 13825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "ea6b6e0dbc0745eda94e9f41993825d5"
format_stamp: "Formatted at 2026-08-06 19:34:58 on dist-test-slave-ss4n"
server_key: "68d06249bc5e497f1faee9ff611d1c1c"
server_key_iv: "53bc22e850c0871b49c15a4422a0b16c"
server_key_version: "encryptionkey@0"
I20260806 19:34:58.855500 13825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "ea6b6e0dbc0745eda94e9f41993825d5"
format_stamp: "Formatted at 2026-08-06 19:34:58 on dist-test-slave-ss4n"
server_key: "68d06249bc5e497f1faee9ff611d1c1c"
server_key_iv: "53bc22e850c0871b49c15a4422a0b16c"
server_key_version: "encryptionkey@0"
I20260806 19:34:58.862309 13825 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 19:34:58.867908 13840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:58.870311 13825 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 19:34:58.870570 13825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "ea6b6e0dbc0745eda94e9f41993825d5"
format_stamp: "Formatted at 2026-08-06 19:34:58 on dist-test-slave-ss4n"
server_key: "68d06249bc5e497f1faee9ff611d1c1c"
server_key_iv: "53bc22e850c0871b49c15a4422a0b16c"
server_key_version: "encryptionkey@0"
I20260806 19:34:58.870890 13825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:58.888988 13825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:58.894819 13825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:58.895211 13825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:58.896391 13825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:34:58.898201 13825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:58.898324 13825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:58.898497 13825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:58.898588 13825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:58.942960 13825 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:44355
I20260806 19:34:58.943005 13960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:44355 every 8 connection(s)
I20260806 19:34:58.944602 13825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:34:58.946439 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 13825
I20260806 19:34:58.946702 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:34:58.947247 10340 external_mini_cluster.cc:1428] Setting key 42fa4863967463553584c3d54b373636
Aug 06 19:34:58 dist-test-slave-ss4n krb5kdc[13576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044898, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
I20260806 19:34:58.977329 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:58.948069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:40363 (local address 127.10.25.62:33227)
0806 19:34:58.948659 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:58.948666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:58.950375 (+ 1709us) server_negotiation.cc:407] Connection header received
0806 19:34:58.952488 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:58.952500 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:58.952671 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:58.953007 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:58.956056 (+ 3049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:58.957402 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:58.960270 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:58.960798 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:58.961033 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:58.966466 (+ 5433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:58.966474 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:58.966478 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:58.966524 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:58.970758 (+ 4234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:58.972889 (+ 2131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:58.972896 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:58.972899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:58.973132 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:58.973992 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:58.973996 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:58.973999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:58.974514 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:58.974788 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:58.976628 (+ 1840us) server_negotiation.cc:299] Negotiation successful
0806 19:34:58.977011 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":319,"threads_started":1}
I20260806 19:34:58.978372 13965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:58.948890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:33227 (local address 127.10.25.2:40363)
0806 19:34:58.950187 (+ 1297us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:58.950242 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:58.952046 (+ 1804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:58.953353 (+ 1307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:58.953364 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:58.954667 (+ 1303us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:58.955669 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:58.955701 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:58.957768 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:58.957774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:58.959955 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:58.959989 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:58.961134 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:58.961143 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:58.961455 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:58.962768 (+ 1313us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:58.962799 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:58.966054 (+ 3255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:58.971206 (+ 5152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:58.971226 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:58.971259 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:58.972605 (+ 1346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:58.973302 (+ 697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:58.973308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:58.973312 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:58.973766 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:58.974777 (+ 1011us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:58.974802 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:58.975788 (+ 986us) client_negotiation.cc:769] Sending connection context
0806 19:34:58.976302 (+ 514us) client_negotiation.cc:240] Negotiation successful
0806 19:34:58.976924 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"thread_start_us":616,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
I20260806 19:34:58.980417 13961 heartbeater.cc:347] Connected to a master server at 127.10.25.62:33227
I20260806 19:34:58.981058 13961 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:58.982246 13961 heartbeater.cc:511] Master 127.10.25.62:33227 requested a full tablet report, sending...
I20260806 19:34:58.984670 13623 ts_manager.cc:194] Registered new tserver with Master: ea6b6e0dbc0745eda94e9f41993825d5 (127.10.25.2:44355)
I20260806 19:34:58.986280 13623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:40363
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:34:59.011030 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:33227
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:34941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:34:59.374817 13819 heartbeater.cc:503] Master 127.10.25.62:33227 was elected leader, sending a full tablet report...
W20260806 19:34:59.402534 13968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:34:59.402860 13968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:34:59.402972 13968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:34:59.411499 13968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:34:59.411660 13968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:34:59.411793 13968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:34:59.422940 13968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:34941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:33227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:34:59.424484 13968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:34:59.425855 13968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:34:59.440461 13974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:59.440465 13975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:34:59.440872 13977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:34:59.441434 13968 server_base.cc:1034] running on GCE node
I20260806 19:34:59.442178 13968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:34:59.443203 13968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:34:59.444518 13968 hybrid_clock.cc:648] HybridClock initialized: now 1786044899444425 us; error 88 us; skew 500 ppm
I20260806 19:34:59.444891 13968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:34:59 dist-test-slave-ss4n krb5kdc[13576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044899, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:34:59.448940 13968 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:34:59.451206 13968 webserver.cc:460] Webserver started at http://127.10.25.3:46215/ using document root <none> and password file <none>
I20260806 19:34:59.451974 13968 fs_manager.cc:359] Metadata directory not provided
I20260806 19:34:59.452090 13968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:34:59.452797 13968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:34:59.456157 13968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "2197727849a04442a2bc7d6006fb1b73"
format_stamp: "Formatted at 2026-08-06 19:34:59 on dist-test-slave-ss4n"
server_key: "adeb6875dc0b7875392f2c53498e830c"
server_key_iv: "b563762e05dbb13adec388d0dd070c91"
server_key_version: "encryptionkey@0"
I20260806 19:34:59.457056 13968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "2197727849a04442a2bc7d6006fb1b73"
format_stamp: "Formatted at 2026-08-06 19:34:59 on dist-test-slave-ss4n"
server_key: "adeb6875dc0b7875392f2c53498e830c"
server_key_iv: "b563762e05dbb13adec388d0dd070c91"
server_key_version: "encryptionkey@0"
I20260806 19:34:59.463219 13968 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.004s
I20260806 19:34:59.467991 13984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:59.470400 13968 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 19:34:59.470710 13968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "2197727849a04442a2bc7d6006fb1b73"
format_stamp: "Formatted at 2026-08-06 19:34:59 on dist-test-slave-ss4n"
server_key: "adeb6875dc0b7875392f2c53498e830c"
server_key_iv: "b563762e05dbb13adec388d0dd070c91"
server_key_version: "encryptionkey@0"
I20260806 19:34:59.471036 13968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:34:59.501644 13968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:34:59.507481 13968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:34:59.507867 13968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:34:59.509109 13968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:34:59.510847 13968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:34:59.510959 13968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:59.511216 13968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:34:59.511314 13968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:34:59.553221 13968 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:38927
I20260806 19:34:59.553282 14099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:38927 every 8 connection(s)
I20260806 19:34:59.554738 13968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:34:59.561975 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 13968
I20260806 19:34:59.562186 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:34:59.562855 10340 external_mini_cluster.cc:1428] Setting key 87c1425ff621525f1305067963a4a926
Aug 06 19:34:59 dist-test-slave-ss4n krb5kdc[13576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044899, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:59.582070 14102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:59.557846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:37247 (local address 127.10.25.62:33227)
0806 19:34:59.558404 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:59.558409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:59.559200 (+ 791us) server_negotiation.cc:407] Connection header received
0806 19:34:59.561452 (+ 2252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:59.561459 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:59.561640 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:59.561787 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:59.563938 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:59.564969 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:59.567209 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:59.567701 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:59.567850 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:59.572591 (+ 4741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:59.572598 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:59.572601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:59.572642 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:59.577045 (+ 4403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:59.578809 (+ 1764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:59.578822 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:59.578825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:59.579036 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:59.579609 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:59.579614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:59.579616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:59.580150 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:59.580531 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:59.581343 (+ 812us) server_negotiation.cc:299] Negotiation successful
0806 19:34:59.581662 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":304,"threads_started":1}
I20260806 19:34:59.583343 14103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:59.558243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:33227 (local address 127.10.25.3:37247)
0806 19:34:59.559023 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0806 19:34:59.559070 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 19:34:59.560973 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:34:59.562115 (+ 1142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:34:59.562132 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:34:59.562828 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:34:59.563698 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:59.563733 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:59.565166 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:59.565171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:59.567025 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:34:59.567040 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:34:59.567978 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:34:59.567988 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:34:59.568258 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:59.569345 (+ 1087us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:34:59.569378 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:34:59.572282 (+ 2904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:34:59.577408 (+ 5126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:59.577426 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:59.577446 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:59.578575 (+ 1129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:59.579190 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:34:59.579197 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:34:59.579201 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:34:59.579460 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:34:59.580479 (+ 1019us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:34:59.580489 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:34:59.580914 (+ 425us) client_negotiation.cc:769] Sending connection context
0806 19:34:59.581228 (+ 314us) client_negotiation.cc:240] Negotiation successful
0806 19:34:59.581855 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":305,"threads_started":1}
I20260806 19:34:59.585165 14100 heartbeater.cc:347] Connected to a master server at 127.10.25.62:33227
I20260806 19:34:59.585594 14100 heartbeater.cc:464] Registering TS with master...
I20260806 19:34:59.586432 14100 heartbeater.cc:511] Master 127.10.25.62:33227 requested a full tablet report, sending...
I20260806 19:34:59.588171 13623 ts_manager.cc:194] Registered new tserver with Master: 2197727849a04442a2bc7d6006fb1b73 (127.10.25.3:38927)
I20260806 19:34:59.589287 13623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:37247
I20260806 19:34:59.593899 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 19:34:59 dist-test-slave-ss4n krb5kdc[13576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044899, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 19:34:59 dist-test-slave-ss4n krb5kdc[13576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044899, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:34:59.662421 14102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:59.643252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44232 (local address 127.10.25.62:33227)
0806 19:34:59.643881 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:59.643888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:59.644132 (+ 244us) server_negotiation.cc:407] Connection header received
0806 19:34:59.644374 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:59.644385 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:59.644483 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:59.644589 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:59.646191 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:59.647175 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:59.648783 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:59.649222 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:59.649333 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:59.654959 (+ 5626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:59.654967 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:59.654972 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:59.655033 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:59.658188 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:59.659698 (+ 1510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:59.659703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:59.659706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:59.659885 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:59.660356 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:59.660359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:59.660362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:59.660843 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:59.661376 (+ 533us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:59.661637 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 19:34:59.661913 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
W20260806 19:34:59.663187 13623 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44232
I20260806 19:34:59.665062 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 13687
Aug 06 19:34:59 dist-test-slave-ss4n krb5kdc[13576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044899, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:34:59.688639 14112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:59.667162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34066 (local address 127.10.25.1:46487)
0806 19:34:59.668127 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:59.668134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:59.668162 (+ 28us) server_negotiation.cc:407] Connection header received
0806 19:34:59.668220 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:59.668226 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:59.668576 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:59.668728 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:59.670370 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:59.671883 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:59.673587 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:59.674651 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:59.674794 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:59.680478 (+ 5684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:59.680497 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:59.680514 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:59.680562 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:59.684634 (+ 4072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:59.685867 (+ 1233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:59.685876 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:59.685879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:59.686043 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:59.686822 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:59.686828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:59.686830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:59.687275 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:59.687584 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:59.688011 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 19:34:59.688276 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":770,"thread_start_us":605,"threads_started":1}
W20260806 19:34:59.690585 13813 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34066
W20260806 19:34:59.692306 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 13687: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:34:59.692579 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 13687
I20260806 19:34:59.713248 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 13825
Aug 06 19:34:59 dist-test-slave-ss4n krb5kdc[13576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044899, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:34:59.733371 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:59.714466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32888 (local address 127.10.25.2:44355)
0806 19:34:59.715040 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:59.715048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:59.715075 (+ 27us) server_negotiation.cc:407] Connection header received
0806 19:34:59.715140 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:59.715147 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:59.715368 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:59.715553 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:59.716883 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:59.718007 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:59.719316 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:59.719888 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:59.720045 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:59.725004 (+ 4959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:59.725032 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:59.725058 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:59.725121 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:59.729583 (+ 4462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:59.730479 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:59.730488 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:59.730490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:59.730663 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:59.731382 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:59.731388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:59.731391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:59.731881 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:59.732180 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:59.732648 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 19:34:59.732920 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":264,"threads_started":1}
W20260806 19:34:59.735594 13956 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:32888
W20260806 19:34:59.737109 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 13825: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:34:59.737293 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 13825
I20260806 19:34:59.758006 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 13968
Aug 06 19:34:59 dist-test-slave-ss4n krb5kdc[13576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044899, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:34:59.778375 14114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:34:59.759388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48668 (local address 127.10.25.3:38927)
0806 19:34:59.760079 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0806 19:34:59.760086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:34:59.760120 (+ 34us) server_negotiation.cc:407] Connection header received
0806 19:34:59.760186 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:34:59.760193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:34:59.760448 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:34:59.760688 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:34:59.762330 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:59.763537 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:59.765036 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:34:59.765790 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:34:59.765950 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:34:59.771260 (+ 5310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:34:59.771280 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:34:59.771299 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:34:59.771355 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:34:59.775041 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:59.775918 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:59.775926 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:59.775929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:59.776073 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:34:59.776543 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:34:59.776548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:34:59.776550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:34:59.776967 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:34:59.777326 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0806 19:34:59.777666 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 19:34:59.777987 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":364,"threads_started":1}
W20260806 19:34:59.780125 14095 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48668
W20260806 19:34:59.781615 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 13968: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:34:59.781848 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 13968
I20260806 19:34:59.802279 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 13589
I20260806 19:34:59.803859 13654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:34:59.915668 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 13589
2026-08-06T19:34:59Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4301 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:35:00 dist-test-slave-ss4n krb5kdc[14118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:35:00 dist-test-slave-ss4n krb5kdc[14118](info): set up 2 sockets
Aug 06 19:35:00 dist-test-slave-ss4n krb5kdc[14118](info): commencing operation
krb5kdc: starting...
W20260806 19:35:01.136226 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:35:01 dist-test-slave-ss4n krb5kdc[14118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:35:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:35:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:35:01.419657 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:32817
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:32817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:01.811192 14131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:01.811569 14131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:01.811678 14131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:01.820395 14131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:35:01.820622 14131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:01.820685 14131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:01.820755 14131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:35:01.820816 14131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:35:01.834014 14131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:32817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:32817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:01.835716 14131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:01.837352 14131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:01.852174 14137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:01.852818 14139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:01.852744 14136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:01.854038 14131 server_base.cc:1034] running on GCE node
I20260806 19:35:01.855461 14131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:01.857857 14131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:01.859174 14131 hybrid_clock.cc:648] HybridClock initialized: now 1786044901859089 us; error 77 us; skew 500 ppm
I20260806 19:35:01.859711 14131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:01 dist-test-slave-ss4n krb5kdc[14118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044901, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:01.867393 14131 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:35:01.869609 14131 webserver.cc:460] Webserver started at http://127.10.25.62:41499/ using document root <none> and password file <none>
I20260806 19:35:01.870496 14131 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:01.870621 14131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:01.870973 14131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:01.875845 14131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "f6feec4c180f4fcbbe12e7bbada93449"
format_stamp: "Formatted at 2026-08-06 19:35:01 on dist-test-slave-ss4n"
server_key: "8841bc6f6bb50d1156535c407e0f0510"
server_key_iv: "75200cc8df7443ba5470b2202cf6a447"
server_key_version: "encryptionkey@0"
I20260806 19:35:01.877152 14131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "f6feec4c180f4fcbbe12e7bbada93449"
format_stamp: "Formatted at 2026-08-06 19:35:01 on dist-test-slave-ss4n"
server_key: "8841bc6f6bb50d1156535c407e0f0510"
server_key_iv: "75200cc8df7443ba5470b2202cf6a447"
server_key_version: "encryptionkey@0"
I20260806 19:35:01.884262 14131 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260806 19:35:01.889055 14146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:01.890825 14131 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 19:35:01.891073 14131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "f6feec4c180f4fcbbe12e7bbada93449"
format_stamp: "Formatted at 2026-08-06 19:35:01 on dist-test-slave-ss4n"
server_key: "8841bc6f6bb50d1156535c407e0f0510"
server_key_iv: "75200cc8df7443ba5470b2202cf6a447"
server_key_version: "encryptionkey@0"
I20260806 19:35:01.891644 14131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:01.918167 14131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:01.925148 14131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:01.925561 14131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:01.953073 14131 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:32817
I20260806 19:35:01.953215 14200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:32817 every 8 connection(s)
I20260806 19:35:01.954965 14131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:35:01.959116 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 14131
I20260806 19:35:01.959383 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:35:01.959723 14201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:01.960258 10340 external_mini_cluster.cc:1428] Setting key a26b9645419f273b7c79766a54252f3a
I20260806 19:35:01.979238 14201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449: Bootstrap starting.
Aug 06 19:35:01 dist-test-slave-ss4n krb5kdc[14118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:35:01.985423 14201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:01.987728 14201 log.cc:824] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:01.993238 14201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449: No bootstrap required, opened a new log
I20260806 19:35:01.996923 14205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:01.963496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52732 (local address 127.10.25.62:32817)
0806 19:35:01.964832 (+ 1336us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:01.964854 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:01.964931 (+ 77us) server_negotiation.cc:407] Connection header received
0806 19:35:01.966926 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:01.966968 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:01.967710 (+ 742us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:01.968320 (+ 610us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:01.970258 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:01.972284 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:01.973993 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:01.976038 (+ 2045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:01.976417 (+ 379us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:01.982054 (+ 5637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:01.982070 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:01.982097 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:01.982205 (+ 108us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:01.987814 (+ 5609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:01.989297 (+ 1483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:01.989307 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:01.989317 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:01.989700 (+ 383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:01.990186 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:01.990190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:01.990192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:01.991547 (+ 1355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:01.992474 (+ 927us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:01.994573 (+ 2099us) server_negotiation.cc:299] Negotiation successful
0806 19:35:01.995377 (+ 804us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":956,"thread_start_us":578,"threads_started":1}
I20260806 19:35:02.005200 14201 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6feec4c180f4fcbbe12e7bbada93449" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 32817 } }
I20260806 19:35:02.005829 14201 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:02.005959 14201 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6feec4c180f4fcbbe12e7bbada93449, State: Initialized, Role: FOLLOWER
I20260806 19:35:02.006857 14201 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6feec4c180f4fcbbe12e7bbada93449" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 32817 } }
I20260806 19:35:02.007184 14201 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:35:02.007364 14201 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:35:02.007573 14201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:02.010993 14201 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6feec4c180f4fcbbe12e7bbada93449" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 32817 } }
I20260806 19:35:02.011855 14201 leader_election.cc:302] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6feec4c180f4fcbbe12e7bbada93449; no voters:
I20260806 19:35:02.012686 14201 leader_election.cc:288] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:35:02.012876 14207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:02.013760 14207 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [term 1 LEADER]: Becoming Leader. State: Replica: f6feec4c180f4fcbbe12e7bbada93449, State: Running, Role: LEADER
I20260806 19:35:02.014773 14207 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6feec4c180f4fcbbe12e7bbada93449" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 32817 } }
I20260806 19:35:02.016072 14201 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:35:02.018891 14210 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6feec4c180f4fcbbe12e7bbada93449. Latest consensus state: current_term: 1 leader_uuid: "f6feec4c180f4fcbbe12e7bbada93449" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6feec4c180f4fcbbe12e7bbada93449" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 32817 } } }
I20260806 19:35:02.019976 14210 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:02.021673 14208 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6feec4c180f4fcbbe12e7bbada93449" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6feec4c180f4fcbbe12e7bbada93449" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 32817 } } }
I20260806 19:35:02.022523 14208 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:02.022778 14214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:35:02.030769 14214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:35:02.047585 14214 catalog_manager.cc:1347] Generated new cluster ID: eec2a46741254960b425a346001614b7
I20260806 19:35:02.047837 14214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:35:02.063553 14214 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:35:02.065721 14214 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:35:02.073995 14214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6feec4c180f4fcbbe12e7bbada93449: Generated new TSK 0
I20260806 19:35:02.076953 14214 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:35:02.159749 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:32817
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:02.598440 14229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:02.598915 14229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:02.599114 14229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:02.607139 14229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:02.607303 14229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:02.607518 14229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:35:02.618304 14229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:32817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:02.619846 14229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:02.621304 14229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:02.639151 14235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:02.639369 14237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:02.640077 14234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:02.640076 14229 server_base.cc:1034] running on GCE node
I20260806 19:35:02.640978 14229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:02.642328 14229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:02.643630 14229 hybrid_clock.cc:648] HybridClock initialized: now 1786044902643575 us; error 48 us; skew 500 ppm
I20260806 19:35:02.644038 14229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:02 dist-test-slave-ss4n krb5kdc[14118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044902, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:02.648336 14229 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:35:02.650954 14229 webserver.cc:460] Webserver started at http://127.10.25.1:40991/ using document root <none> and password file <none>
I20260806 19:35:02.651906 14229 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:02.652096 14229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:02.652626 14229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:02.657783 14229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "89e80c8ce75c4bdfa4368125873d5efa"
format_stamp: "Formatted at 2026-08-06 19:35:02 on dist-test-slave-ss4n"
server_key: "2960e90a60d58ae5b424bc764a1526f2"
server_key_iv: "61ecc39e0caf1118e1a4923749f79336"
server_key_version: "encryptionkey@0"
I20260806 19:35:02.658783 14229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "89e80c8ce75c4bdfa4368125873d5efa"
format_stamp: "Formatted at 2026-08-06 19:35:02 on dist-test-slave-ss4n"
server_key: "2960e90a60d58ae5b424bc764a1526f2"
server_key_iv: "61ecc39e0caf1118e1a4923749f79336"
server_key_version: "encryptionkey@0"
I20260806 19:35:02.666282 14229 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 19:35:02.671849 14244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:02.673691 14229 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
I20260806 19:35:02.673930 14229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "89e80c8ce75c4bdfa4368125873d5efa"
format_stamp: "Formatted at 2026-08-06 19:35:02 on dist-test-slave-ss4n"
server_key: "2960e90a60d58ae5b424bc764a1526f2"
server_key_iv: "61ecc39e0caf1118e1a4923749f79336"
server_key_version: "encryptionkey@0"
I20260806 19:35:02.674207 14229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:02.700403 14229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:02.707468 14229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:02.707852 14229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:02.709013 14229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:02.710742 14229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:02.710868 14229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:02.711035 14229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:02.711122 14229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:02.753937 14229 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:37265
I20260806 19:35:02.754070 14363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:37265 every 8 connection(s)
I20260806 19:35:02.755618 14229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:35:02.766018 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 14229
I20260806 19:35:02.766359 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:35:02.767117 10340 external_mini_cluster.cc:1428] Setting key 034ac3204affa0cf9e0e965c603f0cd8
Aug 06 19:35:02 dist-test-slave-ss4n krb5kdc[14118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044902, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:35:02.784240 14366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:02.758902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:33277 (local address 127.10.25.62:32817)
0806 19:35:02.759398 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:02.759402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:02.760717 (+ 1315us) server_negotiation.cc:407] Connection header received
0806 19:35:02.764022 (+ 3305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:02.764029 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:02.764145 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:02.764280 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:02.766481 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:02.767363 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:02.769287 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:02.769931 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:02.770085 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:02.775162 (+ 5077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:02.775176 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:02.775185 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:02.775252 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:02.779862 (+ 4610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:02.781296 (+ 1434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:02.781301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:02.781304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:02.781461 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:02.782153 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:02.782157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:02.782160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:02.782490 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:02.782666 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:02.783693 (+ 1027us) server_negotiation.cc:299] Negotiation successful
0806 19:35:02.783922 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":281,"threads_started":1}
I20260806 19:35:02.786095 14367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:02.759356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:32817 (local address 127.10.25.1:33277)
0806 19:35:02.760273 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:02.760317 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:02.763604 (+ 3287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:02.764596 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:02.764608 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:02.765242 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:02.766182 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:02.766221 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:02.767552 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:02.767557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:02.769101 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:02.769118 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:02.770219 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:02.770226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:02.770467 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:02.771285 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:02.771312 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:02.774719 (+ 3407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:02.780110 (+ 5391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:02.780126 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:02.780149 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:02.781082 (+ 933us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:02.781686 (+ 604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:02.781694 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:02.781701 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:02.782002 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:02.782647 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:02.782659 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:02.783195 (+ 536us) client_negotiation.cc:769] Sending connection context
0806 19:35:02.783533 (+ 338us) client_negotiation.cc:240] Negotiation successful
0806 19:35:02.784241 (+ 708us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":395,"threads_started":1}
I20260806 19:35:02.788196 14364 heartbeater.cc:347] Connected to a master server at 127.10.25.62:32817
I20260806 19:35:02.788625 14364 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:02.789469 14364 heartbeater.cc:511] Master 127.10.25.62:32817 requested a full tablet report, sending...
I20260806 19:35:02.792105 14165 ts_manager.cc:194] Registered new tserver with Master: 89e80c8ce75c4bdfa4368125873d5efa (127.10.25.1:37265)
I20260806 19:35:02.793773 14165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:33277
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:35:02.831314 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:32817
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:03.213966 14370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:03.214300 14370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:03.214398 14370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:03.222543 14370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:03.222697 14370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:03.222818 14370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:35:03.234925 14370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:32817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:03.236330 14370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:03.237958 14370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:03.253628 14378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:03.253666 14376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:03.253762 14375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:03.255012 14370 server_base.cc:1034] running on GCE node
I20260806 19:35:03.255875 14370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:03.257014 14370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:03.258275 14370 hybrid_clock.cc:648] HybridClock initialized: now 1786044903258194 us; error 64 us; skew 500 ppm
I20260806 19:35:03.258733 14370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:03 dist-test-slave-ss4n krb5kdc[14118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044903, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:03.269412 14370 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:35:03.271519 14370 webserver.cc:460] Webserver started at http://127.10.25.2:37495/ using document root <none> and password file <none>
I20260806 19:35:03.272787 14370 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:03.273020 14370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:03.273567 14370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:03.277149 14370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "4efe099b4df44ae0b8532ed5b4a1646a"
format_stamp: "Formatted at 2026-08-06 19:35:03 on dist-test-slave-ss4n"
server_key: "bfde6f740b5799d8ae09a1adcd7c9c4f"
server_key_iv: "6d777d2fb99affd71beaf0ebdbb1a5e3"
server_key_version: "encryptionkey@0"
I20260806 19:35:03.278231 14370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "4efe099b4df44ae0b8532ed5b4a1646a"
format_stamp: "Formatted at 2026-08-06 19:35:03 on dist-test-slave-ss4n"
server_key: "bfde6f740b5799d8ae09a1adcd7c9c4f"
server_key_iv: "6d777d2fb99affd71beaf0ebdbb1a5e3"
server_key_version: "encryptionkey@0"
I20260806 19:35:03.285059 14370 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:35:03.290776 14385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:03.292819 14370 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260806 19:35:03.293318 14370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "4efe099b4df44ae0b8532ed5b4a1646a"
format_stamp: "Formatted at 2026-08-06 19:35:03 on dist-test-slave-ss4n"
server_key: "bfde6f740b5799d8ae09a1adcd7c9c4f"
server_key_iv: "6d777d2fb99affd71beaf0ebdbb1a5e3"
server_key_version: "encryptionkey@0"
I20260806 19:35:03.293813 14370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:03.317752 14370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:03.323733 14370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:03.324415 14370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:03.326138 14370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:03.328022 14370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:03.328125 14370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:03.328302 14370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:03.328393 14370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:03.373243 14370 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:45957
I20260806 19:35:03.373293 14502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:45957 every 8 connection(s)
I20260806 19:35:03.375802 14370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:35:03.380622 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 14370
I20260806 19:35:03.381163 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:35:03.381893 10340 external_mini_cluster.cc:1428] Setting key 95f4455e217db3f284238b87e756b665
Aug 06 19:35:03 dist-test-slave-ss4n krb5kdc[14118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044903, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:35:03.407251 14505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:03.379760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:60683 (local address 127.10.25.62:32817)
0806 19:35:03.380536 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:03.380546 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:03.381875 (+ 1329us) server_negotiation.cc:407] Connection header received
0806 19:35:03.384344 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:03.384351 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:03.384439 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:03.384569 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:03.387346 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:03.388157 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:03.391076 (+ 2919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:03.391757 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:03.391931 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:03.397009 (+ 5078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:03.397022 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:03.397031 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:03.397094 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:03.401981 (+ 4887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:03.403669 (+ 1688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:03.403676 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:03.403681 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:03.403903 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:03.404635 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:03.404643 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:03.404650 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:03.405177 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:03.405396 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:03.406409 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0806 19:35:03.406750 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":358,"threads_started":1}
I20260806 19:35:03.407948 14506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:03.380347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:32817 (local address 127.10.25.2:60683)
0806 19:35:03.381634 (+ 1287us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:03.381702 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:03.383979 (+ 2277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:03.384939 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:03.384952 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:03.385917 (+ 965us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:03.386886 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:03.386920 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:03.388369 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:03.388378 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:03.390754 (+ 2376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:03.390777 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:03.392116 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:03.392126 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:03.392513 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:03.393646 (+ 1133us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:03.393683 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:03.396562 (+ 2879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:03.402259 (+ 5697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:03.402269 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:03.402286 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:03.403345 (+ 1059us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:03.404118 (+ 773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:03.404123 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:03.404126 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:03.404434 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:03.405423 (+ 989us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:03.405434 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:03.405934 (+ 500us) client_negotiation.cc:769] Sending connection context
0806 19:35:03.406222 (+ 288us) client_negotiation.cc:240] Negotiation successful
0806 19:35:03.406715 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"thread_start_us":577,"threads_started":1}
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
I20260806 19:35:03.409890 14503 heartbeater.cc:347] Connected to a master server at 127.10.25.62:32817
I20260806 19:35:03.410205 14503 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:03.411026 14503 heartbeater.cc:511] Master 127.10.25.62:32817 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
I20260806 19:35:03.412890 14165 ts_manager.cc:194] Registered new tserver with Master: 4efe099b4df44ae0b8532ed5b4a1646a (127.10.25.2:45957)
I20260806 19:35:03.414032 14165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:60683
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:35:03.446776 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:32817
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:35:03.797348 14364 heartbeater.cc:503] Master 127.10.25.62:32817 was elected leader, sending a full tablet report...
W20260806 19:35:03.851369 14509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:03.851692 14509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:03.851846 14509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:03.859789 14509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:03.859941 14509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:03.860118 14509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:35:03.870738 14509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:32817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:03.872303 14509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:03.873980 14509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:03.890822 14515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:03.890866 14517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:03.890939 14514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:03.891942 14509 server_base.cc:1034] running on GCE node
I20260806 19:35:03.892686 14509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:03.894241 14509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:03.895852 14509 hybrid_clock.cc:648] HybridClock initialized: now 1786044903895693 us; error 132 us; skew 500 ppm
I20260806 19:35:03.896410 14509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:03 dist-test-slave-ss4n krb5kdc[14118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044903, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:03.902015 14509 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:35:03.904554 14509 webserver.cc:460] Webserver started at http://127.10.25.3:33751/ using document root <none> and password file <none>
I20260806 19:35:03.905704 14509 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:03.905882 14509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:03.906329 14509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:03.910564 14509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "a68b1a43abb54adaa6d57bd4aa0fd3fb"
format_stamp: "Formatted at 2026-08-06 19:35:03 on dist-test-slave-ss4n"
server_key: "93a899bc31cfad6f13877b347e13ed69"
server_key_iv: "13c49c31c0c2d2a1b54377f51b3df078"
server_key_version: "encryptionkey@0"
I20260806 19:35:03.911646 14509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "a68b1a43abb54adaa6d57bd4aa0fd3fb"
format_stamp: "Formatted at 2026-08-06 19:35:03 on dist-test-slave-ss4n"
server_key: "93a899bc31cfad6f13877b347e13ed69"
server_key_iv: "13c49c31c0c2d2a1b54377f51b3df078"
server_key_version: "encryptionkey@0"
I20260806 19:35:03.919538 14509 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 19:35:03.926192 14524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:03.929042 14509 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 19:35:03.929620 14509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "a68b1a43abb54adaa6d57bd4aa0fd3fb"
format_stamp: "Formatted at 2026-08-06 19:35:03 on dist-test-slave-ss4n"
server_key: "93a899bc31cfad6f13877b347e13ed69"
server_key_iv: "13c49c31c0c2d2a1b54377f51b3df078"
server_key_version: "encryptionkey@0"
I20260806 19:35:03.930032 14509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:03.954877 14509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:03.960539 14509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:03.960927 14509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:03.962523 14509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:03.964916 14509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:03.965090 14509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:03.965294 14509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:03.965456 14509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:04.006945 14509 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:35397
I20260806 19:35:04.007016 14639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:35397 every 8 connection(s)
I20260806 19:35:04.008524 14509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:35:04.009771 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 14509
I20260806 19:35:04.009999 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:35:04.010538 10340 external_mini_cluster.cc:1428] Setting key b982b3961be5874539ad511e5439c743
Aug 06 19:35:04 dist-test-slave-ss4n krb5kdc[14118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044903, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:35:04.032042 14642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:04.011984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:39659 (local address 127.10.25.62:32817)
0806 19:35:04.012613 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:04.012617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:04.013679 (+ 1062us) server_negotiation.cc:407] Connection header received
0806 19:35:04.015696 (+ 2017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:04.015702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:04.015795 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:04.015928 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:04.017859 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:04.018699 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:04.020459 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:04.020961 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:04.021113 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:04.025528 (+ 4415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:04.025538 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:04.025543 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:04.025594 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:04.028531 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:04.029409 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:04.029416 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:04.029424 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:04.029624 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:04.030104 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:04.030109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:04.030112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:04.030413 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:04.030588 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:04.031365 (+ 777us) server_negotiation.cc:299] Negotiation successful
0806 19:35:04.031610 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":299,"threads_started":1}
I20260806 19:35:04.033063 14643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:04.012352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:32817 (local address 127.10.25.3:39659)
0806 19:35:04.013446 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:04.013558 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:04.015388 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:04.016226 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:04.016236 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:04.016904 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:04.017627 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:04.017671 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:04.018894 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:04.018900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:04.020275 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:04.020290 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:04.021251 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:04.021259 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:04.021607 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:04.022456 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:04.022493 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:04.025236 (+ 2743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:04.028726 (+ 3490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:04.028737 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:04.028759 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:04.029263 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:04.029752 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:04.029759 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:04.029763 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:04.029977 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:04.030554 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:04.030563 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:04.030972 (+ 409us) client_negotiation.cc:769] Sending connection context
0806 19:35:04.031279 (+ 307us) client_negotiation.cc:240] Negotiation successful
0806 19:35:04.031852 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":410,"threads_started":1}
I20260806 19:35:04.034900 14640 heartbeater.cc:347] Connected to a master server at 127.10.25.62:32817
I20260806 19:35:04.035341 14640 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:04.036278 14640 heartbeater.cc:511] Master 127.10.25.62:32817 requested a full tablet report, sending...
I20260806 19:35:04.038208 14165 ts_manager.cc:194] Registered new tserver with Master: a68b1a43abb54adaa6d57bd4aa0fd3fb (127.10.25.3:35397)
I20260806 19:35:04.039258 14165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:39659
I20260806 19:35:04.040462 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 19:35:04 dist-test-slave-ss4n krb5kdc[14118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 19:35:04 dist-test-slave-ss4n krb5kdc[14118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:35:04.109153 14648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:04.087816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59260 (local address 127.10.25.1:37265)
0806 19:35:04.088803 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:04.088811 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:04.088842 (+ 31us) server_negotiation.cc:407] Connection header received
0806 19:35:04.089164 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:04.089178 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:04.089771 (+ 593us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:04.089985 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:04.092138 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:04.093288 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:04.094905 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:04.095729 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:04.095881 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:04.102540 (+ 6659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:04.102559 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:04.102577 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:04.102624 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:04.105946 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:04.106810 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:04.106818 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:04.106821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:04.106974 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:04.107393 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:04.107398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:04.107403 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:04.107797 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:04.108144 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:04.108453 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 19:35:04.108750 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":743,"threads_started":1}
Aug 06 19:35:04 dist-test-slave-ss4n krb5kdc[14118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:35:04.133561 14642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:04.114889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52734 (local address 127.10.25.62:32817)
0806 19:35:04.115309 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:04.115318 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:04.115727 (+ 409us) server_negotiation.cc:407] Connection header received
0806 19:35:04.115960 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:04.115965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:04.116037 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:04.116150 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:04.117691 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:04.118541 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:04.119848 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:04.120518 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:04.120754 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:04.125731 (+ 4977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:04.125739 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:04.125742 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:04.125799 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:04.129807 (+ 4008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:04.130870 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:04.130876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:04.130878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:04.131036 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:04.131635 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:04.131639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:04.131641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:04.132005 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:04.132208 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:04.132964 (+ 756us) server_negotiation.cc:299] Negotiation successful
0806 19:35:04.133187 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
W20260806 19:35:04.134477 14164 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52734
I20260806 19:35:04.136159 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 14229
I20260806 19:35:04.154726 14648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:04.137634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59262 (local address 127.10.25.1:37265)
0806 19:35:04.137830 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:04.137835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:04.138281 (+ 446us) server_negotiation.cc:407] Connection header received
0806 19:35:04.138438 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:04.138443 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:04.138568 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:04.138704 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:04.140496 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:04.141570 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:04.143362 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:04.143925 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:04.144072 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:04.147684 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:04.147692 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:04.147697 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:04.147781 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:04.151669 (+ 3888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:04.152672 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:04.152678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:04.152681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:04.152843 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:04.153268 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:04.153272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:04.153274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:04.153681 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:04.153902 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:04.154269 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 19:35:04.154463 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 19:35:04.155500 14359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:04.248281 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 14229
I20260806 19:35:04.268816 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 14370
Aug 06 19:35:04 dist-test-slave-ss4n krb5kdc[14118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:35:04.289830 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:04.270485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49222 (local address 127.10.25.2:45957)
0806 19:35:04.271293 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:04.271302 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:04.271334 (+ 32us) server_negotiation.cc:407] Connection header received
0806 19:35:04.271417 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:04.271424 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:04.271730 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:04.272050 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:04.273652 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:04.274842 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:04.276348 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:04.277005 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:04.277180 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:04.282382 (+ 5202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:04.282401 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:04.282418 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:04.282475 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:04.286140 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:04.287013 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:04.287022 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:04.287024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:04.287178 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:04.287649 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:04.287653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:04.287656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:04.288057 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:04.288439 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:04.288958 (+ 519us) server_negotiation.cc:299] Negotiation successful
0806 19:35:04.289378 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":452,"threads_started":1}
I20260806 19:35:04.291441 14497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:04.382464 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 14370
I20260806 19:35:04.407064 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 14509
Aug 06 19:35:04 dist-test-slave-ss4n krb5kdc[14118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:35:04.429347 14655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:04.408420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40676 (local address 127.10.25.3:35397)
0806 19:35:04.409182 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:04.409190 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:04.409227 (+ 37us) server_negotiation.cc:407] Connection header received
0806 19:35:04.409301 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:04.409308 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:04.409754 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:04.409990 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:04.411579 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:04.412689 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:04.414366 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:04.415010 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:04.415176 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:04.420791 (+ 5615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:04.420809 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:04.420827 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:04.420878 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:04.424655 (+ 3777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:04.426195 (+ 1540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:04.426203 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:04.426206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:04.426395 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:04.427067 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:04.427074 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:04.427077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:04.427684 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:04.428183 (+ 499us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:04.428607 (+ 424us) server_negotiation.cc:299] Negotiation successful
0806 19:35:04.428928 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":397,"threads_started":1}
I20260806 19:35:04.431010 14635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:04.521960 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 14509
I20260806 19:35:04.543382 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 14131
I20260806 19:35:04.545169 14196 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:04.656167 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 14131
2026-08-06T19:35:04Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4740 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:35:04 dist-test-slave-ss4n krb5kdc[14660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:35:04 dist-test-slave-ss4n krb5kdc[14660](info): set up 2 sockets
Aug 06 19:35:04 dist-test-slave-ss4n krb5kdc[14660](info): commencing operation
krb5kdc: starting...
W20260806 19:35:05.881373 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.003s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:35:06 dist-test-slave-ss4n krb5kdc[14660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:35:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:35:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:35:06.163810 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:38251
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:38251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:06.577030 14674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:06.577461 14674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:06.577700 14674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:06.585587 14674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:35:06.585780 14674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:06.585897 14674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:06.586005 14674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:35:06.586105 14674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:35:06.598351 14674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:38251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:38251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:06.599888 14674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:06.601375 14674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:06.618921 14682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:06.619002 14680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:06.619457 14679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:06.620129 14674 server_base.cc:1034] running on GCE node
I20260806 19:35:06.621304 14674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:06.624250 14674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:06.625849 14674 hybrid_clock.cc:648] HybridClock initialized: now 1786044906625757 us; error 90 us; skew 500 ppm
I20260806 19:35:06.626374 14674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:06 dist-test-slave-ss4n krb5kdc[14660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044906, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:06.630389 14674 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:35:06.631949 14674 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:06.632061 14674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:06.632450 14674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:06.636019 14674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "18f7d022cffa4682ba302dc27be317b1"
format_stamp: "Formatted at 2026-08-06 19:35:06 on dist-test-slave-ss4n"
server_key: "b1bd76f325179a36d4881aba0b5111f7"
server_key_iv: "9ff59bfe8550be2964040ded446c1135"
server_key_version: "encryptionkey@0"
I20260806 19:35:06.637027 14674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "18f7d022cffa4682ba302dc27be317b1"
format_stamp: "Formatted at 2026-08-06 19:35:06 on dist-test-slave-ss4n"
server_key: "b1bd76f325179a36d4881aba0b5111f7"
server_key_iv: "9ff59bfe8550be2964040ded446c1135"
server_key_version: "encryptionkey@0"
I20260806 19:35:06.645027 14674 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.004s
I20260806 19:35:06.651239 14688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:06.653457 14674 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260806 19:35:06.653847 14674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "18f7d022cffa4682ba302dc27be317b1"
format_stamp: "Formatted at 2026-08-06 19:35:06 on dist-test-slave-ss4n"
server_key: "b1bd76f325179a36d4881aba0b5111f7"
server_key_iv: "9ff59bfe8550be2964040ded446c1135"
server_key_version: "encryptionkey@0"
I20260806 19:35:06.654150 14674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:06.690470 14674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:06.696753 14674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:06.697117 14674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:06.724294 14674 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:38251
I20260806 19:35:06.724644 14740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:38251 every 8 connection(s)
I20260806 19:35:06.725519 14674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:35:06.727514 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 14674
I20260806 19:35:06.727815 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:35:06.728745 10340 external_mini_cluster.cc:1428] Setting key 9b975cd90f3db01cfea23090217b3bdd
I20260806 19:35:06.731784 14741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 19:35:06 dist-test-slave-ss4n krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:35:06.749197 14741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1: Bootstrap starting.
I20260806 19:35:06.755738 14741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:06.757561 14741 log.cc:824] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:06.762372 14741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1: No bootstrap required, opened a new log
I20260806 19:35:06.763180 14744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:06.731401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52300 (local address 127.10.25.62:38251)
0806 19:35:06.732402 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:06.732415 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:06.732466 (+ 51us) server_negotiation.cc:407] Connection header received
0806 19:35:06.734830 (+ 2364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:06.734865 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:06.735410 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:06.735851 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:06.738416 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:06.740369 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:06.742571 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:06.743510 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:06.743714 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:06.750107 (+ 6393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:06.750121 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:06.750145 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:06.750215 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:06.756895 (+ 6680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:06.758118 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:06.758131 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:06.758143 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:06.758497 (+ 354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:06.759184 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:06.759190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:06.759193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:06.760202 (+ 1009us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:06.760538 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:06.761106 (+ 568us) server_negotiation.cc:299] Negotiation successful
0806 19:35:06.761646 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":483,"threads_started":1}
I20260806 19:35:06.772995 14741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f7d022cffa4682ba302dc27be317b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38251 } }
I20260806 19:35:06.773702 14741 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:06.773804 14741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f7d022cffa4682ba302dc27be317b1, State: Initialized, Role: FOLLOWER
I20260806 19:35:06.774753 14741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f7d022cffa4682ba302dc27be317b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38251 } }
I20260806 19:35:06.775189 14741 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:35:06.775406 14741 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:35:06.775686 14741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:06.779942 14741 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f7d022cffa4682ba302dc27be317b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38251 } }
I20260806 19:35:06.780994 14741 leader_election.cc:302] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18f7d022cffa4682ba302dc27be317b1; no voters:
I20260806 19:35:06.781934 14741 leader_election.cc:288] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:35:06.782130 14746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:06.783046 14746 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [term 1 LEADER]: Becoming Leader. State: Replica: 18f7d022cffa4682ba302dc27be317b1, State: Running, Role: LEADER
I20260806 19:35:06.783731 14746 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f7d022cffa4682ba302dc27be317b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38251 } }
I20260806 19:35:06.784614 14741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:35:06.787700 14747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18f7d022cffa4682ba302dc27be317b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f7d022cffa4682ba302dc27be317b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38251 } } }
I20260806 19:35:06.787691 14748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18f7d022cffa4682ba302dc27be317b1. Latest consensus state: current_term: 1 leader_uuid: "18f7d022cffa4682ba302dc27be317b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f7d022cffa4682ba302dc27be317b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38251 } } }
I20260806 19:35:06.788216 14747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:06.788239 14748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:06.793378 14755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:35:06.802597 14755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:35:06.817683 14755 catalog_manager.cc:1347] Generated new cluster ID: c9c10a94d01745848764e2f85d5a5e84
I20260806 19:35:06.817868 14755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:35:06.841104 14755 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:35:06.843678 14755 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:35:06.855681 14755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1: Generated new TSK 0
I20260806 19:35:06.856998 14755 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:35:06.960182 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:38251
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:07.367045 14769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:07.367604 14769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:07.368072 14769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:07.378383 14769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:07.378701 14769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:07.378897 14769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:35:07.392447 14769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:38251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:07.394124 14769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:07.395645 14769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:07.411898 14775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:07.411937 14776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:07.412822 14778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:07.413336 14769 server_base.cc:1034] running on GCE node
I20260806 19:35:07.414194 14769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:07.415448 14769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:07.418921 14769 hybrid_clock.cc:648] HybridClock initialized: now 1786044907418843 us; error 77 us; skew 500 ppm
I20260806 19:35:07.419397 14769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:07 dist-test-slave-ss4n krb5kdc[14660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044907, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:07.424069 14769 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:35:07.425849 14769 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:07.426034 14769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:07.426446 14769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:07.430001 14769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "c0649cf5de3d4a1eb8bf34b9bfc8e75f"
format_stamp: "Formatted at 2026-08-06 19:35:07 on dist-test-slave-ss4n"
server_key: "c56842d5ddacd91c5a2386cff6180af9"
server_key_iv: "23832d1df1a8986107bdaf4ac84bb227"
server_key_version: "encryptionkey@0"
I20260806 19:35:07.431077 14769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "c0649cf5de3d4a1eb8bf34b9bfc8e75f"
format_stamp: "Formatted at 2026-08-06 19:35:07 on dist-test-slave-ss4n"
server_key: "c56842d5ddacd91c5a2386cff6180af9"
server_key_iv: "23832d1df1a8986107bdaf4ac84bb227"
server_key_version: "encryptionkey@0"
I20260806 19:35:07.439570 14769 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260806 19:35:07.444247 14784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:07.446280 14769 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:35:07.446532 14769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "c0649cf5de3d4a1eb8bf34b9bfc8e75f"
format_stamp: "Formatted at 2026-08-06 19:35:07 on dist-test-slave-ss4n"
server_key: "c56842d5ddacd91c5a2386cff6180af9"
server_key_iv: "23832d1df1a8986107bdaf4ac84bb227"
server_key_version: "encryptionkey@0"
I20260806 19:35:07.446815 14769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:07.470171 14769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:07.476140 14769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:07.476478 14769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:07.478013 14769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:07.479820 14769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:07.479982 14769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:07.480121 14769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:07.480217 14769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:07.530514 14769 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:44355
I20260806 19:35:07.530741 14897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:44355 every 8 connection(s)
I20260806 19:35:07.531628 14769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:35:07.533694 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 14769
I20260806 19:35:07.533962 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:35:07.534765 10340 external_mini_cluster.cc:1428] Setting key ef4268fff786f3367009ace5dc3220d3
Aug 06 19:35:07 dist-test-slave-ss4n krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044907, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
I20260806 19:35:07.561478 14900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:07.534805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:54361 (local address 127.10.25.62:38251)
0806 19:35:07.535413 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:07.535420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:07.537211 (+ 1791us) server_negotiation.cc:407] Connection header received
0806 19:35:07.539273 (+ 2062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:07.539279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:07.539375 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:07.539514 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:07.541402 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:07.542271 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:07.545189 (+ 2918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:07.545972 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:07.546234 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:07.552166 (+ 5932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:07.552178 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:07.552187 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:07.552261 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:07.556874 (+ 4613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:07.558230 (+ 1356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:07.558239 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:07.558243 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:07.558678 (+ 435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:07.559293 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:07.559298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:07.559301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:07.559690 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:07.559860 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:07.560706 (+ 846us) server_negotiation.cc:299] Negotiation successful
0806 19:35:07.561063 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":302,"threads_started":1}
I20260806 19:35:07.562958 14901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:07.535404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:38251 (local address 127.10.25.1:54361)
0806 19:35:07.536852 (+ 1448us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:07.537097 (+ 245us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:07.538952 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:07.539791 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:07.539801 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:07.540404 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:07.541139 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:07.541164 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:07.542460 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:07.542465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:07.544653 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:07.544675 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:07.546384 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:07.546391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:07.546666 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:07.547761 (+ 1095us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:07.547798 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:07.551688 (+ 3890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:07.557224 (+ 5536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:07.557235 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:07.557254 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:07.558023 (+ 769us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:07.558838 (+ 815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:07.558842 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:07.558845 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:07.559132 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:07.559856 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:07.559867 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:07.560279 (+ 412us) client_negotiation.cc:769] Sending connection context
0806 19:35:07.560581 (+ 302us) client_negotiation.cc:240] Negotiation successful
0806 19:35:07.561336 (+ 755us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1044,"thread_start_us":763,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
I20260806 19:35:07.565148 14898 heartbeater.cc:347] Connected to a master server at 127.10.25.62:38251
I20260806 19:35:07.565616 14898 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:07.566429 14898 heartbeater.cc:511] Master 127.10.25.62:38251 requested a full tablet report, sending...
I20260806 19:35:07.569680 14706 ts_manager.cc:194] Registered new tserver with Master: c0649cf5de3d4a1eb8bf34b9bfc8e75f (127.10.25.1:44355)
I20260806 19:35:07.571520 14706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:54361
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:35:07.603142 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:38251
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:08.064973 14904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:08.065673 14904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:08.065852 14904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:08.075038 14904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:08.075206 14904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:08.075385 14904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:35:08.088212 14904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:38251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:08.090255 14904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:08.091964 14904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:08.107676 14912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:08.107757 14909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:08.107726 14910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:08.114292 14904 server_base.cc:1034] running on GCE node
I20260806 19:35:08.115051 14904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:08.116426 14904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:08.117720 14904 hybrid_clock.cc:648] HybridClock initialized: now 1786044908117631 us; error 89 us; skew 500 ppm
I20260806 19:35:08.118227 14904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:08 dist-test-slave-ss4n krb5kdc[14660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044908, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:08.122529 14904 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:35:08.124182 14904 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:08.124320 14904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:08.124744 14904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:08.128386 14904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb"
format_stamp: "Formatted at 2026-08-06 19:35:08 on dist-test-slave-ss4n"
server_key: "9714581ea191db6152c7707a156a8bfa"
server_key_iv: "ee200d1c6c788aa683667e2d672e712e"
server_key_version: "encryptionkey@0"
I20260806 19:35:08.129372 14904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb"
format_stamp: "Formatted at 2026-08-06 19:35:08 on dist-test-slave-ss4n"
server_key: "9714581ea191db6152c7707a156a8bfa"
server_key_iv: "ee200d1c6c788aa683667e2d672e712e"
server_key_version: "encryptionkey@0"
I20260806 19:35:08.137092 14904 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.004s
I20260806 19:35:08.142376 14918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:08.144573 14904 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:35:08.144846 14904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb"
format_stamp: "Formatted at 2026-08-06 19:35:08 on dist-test-slave-ss4n"
server_key: "9714581ea191db6152c7707a156a8bfa"
server_key_iv: "ee200d1c6c788aa683667e2d672e712e"
server_key_version: "encryptionkey@0"
I20260806 19:35:08.145152 14904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:08.167361 14904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:08.174238 14904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:08.174748 14904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:08.176034 14904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:08.177695 14904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:08.177842 14904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:08.178002 14904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:08.178100 14904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:08.219861 14904 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:33739
I20260806 19:35:08.219914 15034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:33739 every 8 connection(s)
I20260806 19:35:08.221421 14904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:35:08.223860 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 14904
I20260806 19:35:08.224104 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:35:08.224622 10340 external_mini_cluster.cc:1428] Setting key bd3e72348bbbf14b78ed5a503f40a1d0
Aug 06 19:35:08 dist-test-slave-ss4n krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044908, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:35:08.254166 15037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:08.224928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:51635 (local address 127.10.25.62:38251)
0806 19:35:08.225590 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:08.225598 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:08.226938 (+ 1340us) server_negotiation.cc:407] Connection header received
0806 19:35:08.230109 (+ 3171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:08.230120 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:08.230278 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:08.230481 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:08.233200 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:08.234635 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:08.237696 (+ 3061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:08.238247 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:08.238414 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:08.244937 (+ 6523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:08.244945 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:08.244949 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:08.244996 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:08.249031 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:08.250966 (+ 1935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:08.250975 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:08.250979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:08.251310 (+ 331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:08.251895 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:08.251901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:08.251905 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:08.252299 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:08.252474 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:08.253371 (+ 897us) server_negotiation.cc:299] Negotiation successful
0806 19:35:08.253725 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":308,"threads_started":1}
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
I20260806 19:35:08.255354 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:08.225748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:38251 (local address 127.10.25.2:51635)
0806 19:35:08.226697 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:08.226798 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:08.229595 (+ 2797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:08.230882 (+ 1287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:08.230892 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:08.231576 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:08.232559 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:08.232590 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:08.235047 (+ 2457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:08.235055 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:08.237166 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:08.237207 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:08.238693 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:08.238704 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:08.239217 (+ 513us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:08.240415 (+ 1198us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:08.240474 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:08.244494 (+ 4020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:08.249538 (+ 5044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:08.249557 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:08.249588 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:08.250727 (+ 1139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:08.251476 (+ 749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:08.251489 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:08.251493 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:08.251745 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:08.252469 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:08.252479 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:08.252920 (+ 441us) client_negotiation.cc:769] Sending connection context
0806 19:35:08.253246 (+ 326us) client_negotiation.cc:240] Negotiation successful
0806 19:35:08.253945 (+ 699us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":409,"threads_started":1}
I20260806 19:35:08.257854 15035 heartbeater.cc:347] Connected to a master server at 127.10.25.62:38251
I20260806 19:35:08.258618 15035 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
I20260806 19:35:08.259626 15035 heartbeater.cc:511] Master 127.10.25.62:38251 requested a full tablet report, sending...
I20260806 19:35:08.262128 14706 ts_manager.cc:194] Registered new tserver with Master: ab8ac6f1fe6449a9b329b1345f7a94bb (127.10.25.2:33739)
I20260806 19:35:08.263164 14706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:51635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:35:08.299485 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:38251
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:35:08.575452 14898 heartbeater.cc:503] Master 127.10.25.62:38251 was elected leader, sending a full tablet report...
W20260806 19:35:08.719971 15041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:08.720319 15041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:08.720474 15041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:08.729209 15041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:08.729354 15041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:08.729588 15041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:35:08.741876 15041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:38251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:08.743523 15041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:08.745115 15041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:08.764425 15046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:08.764619 15049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:08.764437 15047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:08.765609 15041 server_base.cc:1034] running on GCE node
I20260806 19:35:08.766774 15041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:08.768322 15041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:08.771847 15041 hybrid_clock.cc:648] HybridClock initialized: now 1786044908771742 us; error 162 us; skew 500 ppm
I20260806 19:35:08.772411 15041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:08 dist-test-slave-ss4n krb5kdc[14660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044908, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:08.782747 15041 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:35:08.784518 15041 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:08.784651 15041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:08.785081 15041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:08.788878 15041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "a87ef90c67ba4512bbab31bd798bc6c4"
format_stamp: "Formatted at 2026-08-06 19:35:08 on dist-test-slave-ss4n"
server_key: "590e351840109e622c885ecfac9d18f5"
server_key_iv: "405dbbd0d6ccfb601ffaee055ede08f6"
server_key_version: "encryptionkey@0"
I20260806 19:35:08.790061 15041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "a87ef90c67ba4512bbab31bd798bc6c4"
format_stamp: "Formatted at 2026-08-06 19:35:08 on dist-test-slave-ss4n"
server_key: "590e351840109e622c885ecfac9d18f5"
server_key_iv: "405dbbd0d6ccfb601ffaee055ede08f6"
server_key_version: "encryptionkey@0"
I20260806 19:35:08.798013 15041 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 19:35:08.802753 15055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:08.804829 15041 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 19:35:08.805064 15041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "a87ef90c67ba4512bbab31bd798bc6c4"
format_stamp: "Formatted at 2026-08-06 19:35:08 on dist-test-slave-ss4n"
server_key: "590e351840109e622c885ecfac9d18f5"
server_key_iv: "405dbbd0d6ccfb601ffaee055ede08f6"
server_key_version: "encryptionkey@0"
I20260806 19:35:08.805346 15041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:08.831501 15041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:08.838768 15041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:08.839136 15041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:08.840415 15041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:08.842193 15041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:08.842335 15041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:08.842666 15041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:08.842828 15041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:08.896564 15041 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:45475
I20260806 19:35:08.896610 15169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:45475 every 8 connection(s)
I20260806 19:35:08.897646 15041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:35:08.903762 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 15041
I20260806 19:35:08.904083 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:35:08.904814 10340 external_mini_cluster.cc:1428] Setting key 73241f326a3ab44806a274e586b732df
Aug 06 19:35:08 dist-test-slave-ss4n krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044908, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:35:08.934652 15172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:08.901441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:45743 (local address 127.10.25.62:38251)
0806 19:35:08.902229 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:08.902236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:08.904038 (+ 1802us) server_negotiation.cc:407] Connection header received
0806 19:35:08.908762 (+ 4724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:08.908769 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:08.908935 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:08.909178 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:08.913589 (+ 4411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:08.914702 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:08.916744 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:08.917774 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:08.918025 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:08.924853 (+ 6828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:08.924860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:08.924866 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:08.924922 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:08.928623 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:08.930583 (+ 1960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:08.930589 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:08.930591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:08.930795 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:08.931855 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:08.931862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:08.931865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:08.932428 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:08.932626 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:08.933945 (+ 1319us) server_negotiation.cc:299] Negotiation successful
0806 19:35:08.934243 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":327,"threads_started":1}
I20260806 19:35:08.938935 15173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:08.902077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:38251 (local address 127.10.25.3:45743)
0806 19:35:08.903658 (+ 1581us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:08.903756 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:08.908148 (+ 4392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:08.909720 (+ 1572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:08.909731 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:08.911263 (+ 1532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:08.913085 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:08.913127 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:08.915033 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:08.915039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:08.916495 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:08.916518 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:08.918251 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:08.918265 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:08.918604 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:08.920018 (+ 1414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:08.920054 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:08.924555 (+ 4501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:08.928987 (+ 4432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:08.929002 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:08.929034 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:08.930042 (+ 1008us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:08.931022 (+ 980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:08.931027 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:08.931031 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:08.931572 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:08.932724 (+ 1152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:08.932733 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:08.933237 (+ 504us) client_negotiation.cc:769] Sending connection context
0806 19:35:08.933662 (+ 425us) client_negotiation.cc:240] Negotiation successful
0806 19:35:08.934221 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"thread_start_us":558,"threads_started":1}
I20260806 19:35:08.941256 15170 heartbeater.cc:347] Connected to a master server at 127.10.25.62:38251
I20260806 19:35:08.941720 15170 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:08.942555 15170 heartbeater.cc:511] Master 127.10.25.62:38251 requested a full tablet report, sending...
I20260806 19:35:08.944347 14706 ts_manager.cc:194] Registered new tserver with Master: a87ef90c67ba4512bbab31bd798bc6c4 (127.10.25.3:45475)
I20260806 19:35:08.945274 14706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:45743
I20260806 19:35:08.955054 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:35:08.983695 15172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:08.961942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52308 (local address 127.10.25.62:38251)
0806 19:35:08.962145 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:08.962150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:08.962687 (+ 537us) server_negotiation.cc:407] Connection header received
0806 19:35:08.962859 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:08.962863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:08.962969 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:08.963109 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:08.965177 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:08.966244 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:08.968257 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:08.968878 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:08.969081 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:08.973688 (+ 4607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:08.973719 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:08.973727 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:08.973798 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:08.978229 (+ 4431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:08.980179 (+ 1950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:08.980187 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:08.980190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:08.980379 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:08.981093 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:08.981102 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:08.981107 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:08.982053 (+ 946us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:08.982328 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:08.983099 (+ 771us) server_negotiation.cc:299] Negotiation successful
0806 19:35:08.983371 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 19:35:08.991808 14706 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:35:09.004510 14706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:35:09.055970 15187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:09.027812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53612 (local address 127.10.25.2:33739)
0806 19:35:09.038856 (+ 11044us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:09.038872 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:09.041404 (+ 2532us) server_negotiation.cc:407] Connection header received
0806 19:35:09.041828 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:09.041841 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:09.042251 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:09.042482 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:09.045596 (+ 3114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.047499 (+ 1903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.051480 (+ 3981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.053862 (+ 2382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.054048 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:09.054324 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:09.055091 (+ 767us) server_negotiation.cc:299] Negotiation successful
0806 19:35:09.055402 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10739,"thread_start_us":11064,"threads_started":1}
I20260806 19:35:09.057027 15184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:09.027164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:44355 (local address 127.0.0.1:39512)
0806 19:35:09.040714 (+ 13550us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:09.040778 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:09.041103 (+ 325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:09.043386 (+ 2283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:09.043397 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:09.043456 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:09.044205 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:09.044227 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.047440 (+ 3213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.047446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:09.051274 (+ 3828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:09.051285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.055644 (+ 4359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.055651 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:09.055902 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:09.055944 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 19:35:09.056084 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 19:35:09.056362 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13031,"mutex_wait_us":283,"thread_start_us":12850,"threads_started":1}
I20260806 19:35:09.057382 15189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:09.041517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48428 (local address 127.10.25.3:45475)
0806 19:35:09.044621 (+ 3104us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:09.044635 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:09.045831 (+ 1196us) server_negotiation.cc:407] Connection header received
0806 19:35:09.046269 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:09.046282 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:09.046601 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:09.046831 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:09.047682 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.049317 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.052819 (+ 3502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.055051 (+ 2232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.055223 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:09.055503 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:09.056472 (+ 969us) server_negotiation.cc:299] Negotiation successful
0806 19:35:09.056696 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2819,"thread_start_us":2522,"threads_started":1}
I20260806 19:35:09.057541 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:09.027449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:33739 (local address 127.0.0.1:53612)
0806 19:35:09.041206 (+ 13757us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:09.041253 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:09.041561 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:09.042976 (+ 1415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:09.042989 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:09.043043 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:09.045240 (+ 2197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:09.045291 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.047708 (+ 2417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.047713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:09.051251 (+ 3538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:09.051277 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.054255 (+ 2978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.054268 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:09.054594 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:09.054677 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 19:35:09.056379 (+ 1702us) client_negotiation.cc:240] Negotiation successful
0806 19:35:09.056592 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13633,"mutex_wait_us":1206,"thread_start_us":12139,"threads_started":1}
I20260806 19:35:09.058069 15186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:09.027462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39512 (local address 127.10.25.1:44355)
0806 19:35:09.040336 (+ 12874us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:09.040346 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:09.040981 (+ 635us) server_negotiation.cc:407] Connection header received
0806 19:35:09.041464 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:09.041474 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:09.041928 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:09.042625 (+ 697us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:09.045181 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.047242 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.052756 (+ 5514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.054881 (+ 2125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.055046 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:09.055428 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:09.056615 (+ 1187us) server_negotiation.cc:299] Negotiation successful
0806 19:35:09.056846 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12692,"thread_start_us":12446,"threads_started":1}
I20260806 19:35:09.058578 15188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:09.041416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:45475 (local address 127.0.0.1:48428)
0806 19:35:09.045470 (+ 4054us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:09.045536 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:09.046003 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:09.046958 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:09.046963 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:09.046994 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:09.047439 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:09.047457 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.049616 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.049625 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:09.052403 (+ 2778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:09.052424 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.055749 (+ 3325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.055757 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:09.056047 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:09.056099 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 19:35:09.057852 (+ 1753us) client_negotiation.cc:240] Negotiation successful
0806 19:35:09.058120 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3872,"mutex_wait_us":42,"thread_start_us":3622,"threads_started":1}
I20260806 19:35:09.063377 14967 tablet_service.cc:1467] Processing CreateTablet for tablet fe020bcd9af5415da777ffab2a8d7547 (DEFAULT_TABLE table=test-table [id=10528d76d988499489726a3d9a34f297]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:09.063568 14832 tablet_service.cc:1467] Processing CreateTablet for tablet fe020bcd9af5415da777ffab2a8d7547 (DEFAULT_TABLE table=test-table [id=10528d76d988499489726a3d9a34f297]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:09.064447 15103 tablet_service.cc:1467] Processing CreateTablet for tablet fe020bcd9af5415da777ffab2a8d7547 (DEFAULT_TABLE table=test-table [id=10528d76d988499489726a3d9a34f297]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:09.065440 14967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe020bcd9af5415da777ffab2a8d7547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:09.065609 14832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe020bcd9af5415da777ffab2a8d7547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:09.065948 15103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe020bcd9af5415da777ffab2a8d7547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:09.087330 15190 tablet_bootstrap.cc:492] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb: Bootstrap starting.
I20260806 19:35:09.090971 15192 tablet_bootstrap.cc:492] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f: Bootstrap starting.
I20260806 19:35:09.092226 15191 tablet_bootstrap.cc:492] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4: Bootstrap starting.
I20260806 19:35:09.093618 15190 tablet_bootstrap.cc:654] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:09.096939 15191 tablet_bootstrap.cc:654] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:09.096928 15190 log.cc:824] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:09.098455 15191 log.cc:824] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:09.099563 15192 tablet_bootstrap.cc:654] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:09.101395 15192 log.cc:824] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:09.102876 15191 tablet_bootstrap.cc:492] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4: No bootstrap required, opened a new log
I20260806 19:35:09.103245 15191 ts_tablet_manager.cc:1397] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260806 19:35:09.105268 15190 tablet_bootstrap.cc:492] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb: No bootstrap required, opened a new log
I20260806 19:35:09.106115 15190 ts_tablet_manager.cc:1397] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.000s
I20260806 19:35:09.106225 15192 tablet_bootstrap.cc:492] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f: No bootstrap required, opened a new log
I20260806 19:35:09.106564 15192 ts_tablet_manager.cc:1397] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f: Time spent bootstrapping tablet: real 0.016s user 0.014s sys 0.000s
I20260806 19:35:09.118053 15190 raft_consensus.cc:348] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0649cf5de3d4a1eb8bf34b9bfc8e75f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 44355 } } peers { permanent_uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33739 } } peers { permanent_uuid: "a87ef90c67ba4512bbab31bd798bc6c4" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45475 } }
I20260806 19:35:09.117846 15191 raft_consensus.cc:348] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0649cf5de3d4a1eb8bf34b9bfc8e75f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 44355 } } peers { permanent_uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33739 } } peers { permanent_uuid: "a87ef90c67ba4512bbab31bd798bc6c4" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45475 } }
I20260806 19:35:09.118592 15190 raft_consensus.cc:374] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:09.118618 15191 raft_consensus.cc:374] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:09.118872 15191 raft_consensus.cc:729] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a87ef90c67ba4512bbab31bd798bc6c4, State: Initialized, Role: FOLLOWER
I20260806 19:35:09.118748 15190 raft_consensus.cc:729] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab8ac6f1fe6449a9b329b1345f7a94bb, State: Initialized, Role: FOLLOWER
I20260806 19:35:09.119859 15190 consensus_queue.cc:260] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0649cf5de3d4a1eb8bf34b9bfc8e75f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 44355 } } peers { permanent_uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33739 } } peers { permanent_uuid: "a87ef90c67ba4512bbab31bd798bc6c4" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45475 } }
I20260806 19:35:09.120699 15191 consensus_queue.cc:260] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0649cf5de3d4a1eb8bf34b9bfc8e75f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 44355 } } peers { permanent_uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33739 } } peers { permanent_uuid: "a87ef90c67ba4512bbab31bd798bc6c4" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45475 } }
I20260806 19:35:09.121366 15192 raft_consensus.cc:348] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0649cf5de3d4a1eb8bf34b9bfc8e75f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 44355 } } peers { permanent_uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33739 } } peers { permanent_uuid: "a87ef90c67ba4512bbab31bd798bc6c4" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45475 } }
I20260806 19:35:09.122290 15192 raft_consensus.cc:374] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:09.122504 15192 raft_consensus.cc:729] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0649cf5de3d4a1eb8bf34b9bfc8e75f, State: Initialized, Role: FOLLOWER
I20260806 19:35:09.122849 15035 heartbeater.cc:503] Master 127.10.25.62:38251 was elected leader, sending a full tablet report...
I20260806 19:35:09.123642 15170 heartbeater.cc:503] Master 127.10.25.62:38251 was elected leader, sending a full tablet report...
I20260806 19:35:09.123550 15192 consensus_queue.cc:260] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0649cf5de3d4a1eb8bf34b9bfc8e75f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 44355 } } peers { permanent_uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33739 } } peers { permanent_uuid: "a87ef90c67ba4512bbab31bd798bc6c4" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45475 } }
I20260806 19:35:09.124498 15191 ts_tablet_manager.cc:1428] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4: Time spent starting tablet: real 0.021s user 0.020s sys 0.000s
I20260806 19:35:09.126715 15190 ts_tablet_manager.cc:1428] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb: Time spent starting tablet: real 0.020s user 0.010s sys 0.006s
I20260806 19:35:09.126806 15192 ts_tablet_manager.cc:1428] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f: Time spent starting tablet: real 0.020s user 0.014s sys 0.004s
W20260806 19:35:09.150534 15171 tablet.cc:2369] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:35:09.224546 15036 tablet.cc:2369] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:35:09.285081 14899 tablet.cc:2369] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:35:09.352394 15196 raft_consensus.cc:482] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:35:09.353185 15196 raft_consensus.cc:504] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0649cf5de3d4a1eb8bf34b9bfc8e75f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 44355 } } peers { permanent_uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33739 } } peers { permanent_uuid: "a87ef90c67ba4512bbab31bd798bc6c4" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45475 } }
I20260806 19:35:09.355274 15196 leader_election.cc:288] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0649cf5de3d4a1eb8bf34b9bfc8e75f (127.10.25.1:44355), a87ef90c67ba4512bbab31bd798bc6c4 (127.10.25.3:45475)
I20260806 19:35:09.366609 15189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:09.358204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:53655 (local address 127.10.25.3:45475)
0806 19:35:09.358418 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:09.358427 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:09.359332 (+ 905us) server_negotiation.cc:407] Connection header received
0806 19:35:09.359878 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:09.359891 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:09.359991 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:09.360559 (+ 568us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:09.361205 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.362048 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.364182 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.365104 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.365296 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:09.365463 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:09.366144 (+ 681us) server_negotiation.cc:299] Negotiation successful
0806 19:35:09.366311 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 19:35:09.366950 15200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:09.357892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:45475 (local address 127.10.25.2:53655)
0806 19:35:09.359189 (+ 1297us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:09.359216 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:09.359403 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:09.360560 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:09.360566 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:09.360596 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:09.361020 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:09.361031 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.362267 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.362280 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:09.363957 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:09.363972 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.365383 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.365391 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:09.365901 (+ 510us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:09.365947 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 19:35:09.366094 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 19:35:09.366266 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1097,"mutex_wait_us":30,"thread_start_us":723,"threads_started":1}
I20260806 19:35:09.367983 15186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:09.356298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:34669 (local address 127.10.25.1:44355)
0806 19:35:09.356986 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:09.356998 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:09.358059 (+ 1061us) server_negotiation.cc:407] Connection header received
0806 19:35:09.358839 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:09.358846 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:09.358941 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:09.358997 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:09.360306 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.361440 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.364217 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.366098 (+ 1881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.366498 (+ 400us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:09.366652 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:09.367416 (+ 764us) server_negotiation.cc:299] Negotiation successful
0806 19:35:09.367699 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 19:35:09.368000 15199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:09.355870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:44355 (local address 127.10.25.2:34669)
0806 19:35:09.357874 (+ 2004us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:09.357900 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:09.358434 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:09.359262 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:09.359266 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:09.359364 (+ 98us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:09.360089 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:09.360102 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.361697 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.361705 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:09.363646 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:09.363667 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.366653 (+ 2986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.366663 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:09.367137 (+ 474us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:09.367194 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 19:35:09.367347 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 19:35:09.367509 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1472,"mutex_wait_us":31,"thread_start_us":880,"threads_started":1}
I20260806 19:35:09.368669 15124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe020bcd9af5415da777ffab2a8d7547" candidate_uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a87ef90c67ba4512bbab31bd798bc6c4" is_pre_election: true
I20260806 19:35:09.369293 15124 raft_consensus.cc:2454] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab8ac6f1fe6449a9b329b1345f7a94bb in term 0.
I20260806 19:35:09.369845 14853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe020bcd9af5415da777ffab2a8d7547" candidate_uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0649cf5de3d4a1eb8bf34b9bfc8e75f" is_pre_election: true
I20260806 19:35:09.370240 14922 leader_election.cc:302] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a87ef90c67ba4512bbab31bd798bc6c4, ab8ac6f1fe6449a9b329b1345f7a94bb; no voters:
I20260806 19:35:09.370957 15196 raft_consensus.cc:2781] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:35:09.371116 15196 raft_consensus.cc:482] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:35:09.371021 14853 raft_consensus.cc:2454] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab8ac6f1fe6449a9b329b1345f7a94bb in term 0.
I20260806 19:35:09.371302 15196 raft_consensus.cc:3037] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:09.374320 15196 raft_consensus.cc:504] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0649cf5de3d4a1eb8bf34b9bfc8e75f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 44355 } } peers { permanent_uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33739 } } peers { permanent_uuid: "a87ef90c67ba4512bbab31bd798bc6c4" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45475 } }
I20260806 19:35:09.375125 15196 leader_election.cc:288] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [CANDIDATE]: Term 1 election: Requested vote from peers c0649cf5de3d4a1eb8bf34b9bfc8e75f (127.10.25.1:44355), a87ef90c67ba4512bbab31bd798bc6c4 (127.10.25.3:45475)
I20260806 19:35:09.375831 14853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe020bcd9af5415da777ffab2a8d7547" candidate_uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0649cf5de3d4a1eb8bf34b9bfc8e75f"
I20260806 19:35:09.376070 14853 raft_consensus.cc:3037] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:09.376313 15124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe020bcd9af5415da777ffab2a8d7547" candidate_uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a87ef90c67ba4512bbab31bd798bc6c4"
I20260806 19:35:09.376596 15124 raft_consensus.cc:3037] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:09.379333 14853 raft_consensus.cc:2454] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab8ac6f1fe6449a9b329b1345f7a94bb in term 1.
I20260806 19:35:09.380172 14922 leader_election.cc:302] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab8ac6f1fe6449a9b329b1345f7a94bb, c0649cf5de3d4a1eb8bf34b9bfc8e75f; no voters:
I20260806 19:35:09.380824 15196 raft_consensus.cc:2781] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:09.382119 15196 raft_consensus.cc:686] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [term 1 LEADER]: Becoming Leader. State: Replica: ab8ac6f1fe6449a9b329b1345f7a94bb, State: Running, Role: LEADER
I20260806 19:35:09.382915 15196 consensus_queue.cc:237] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0649cf5de3d4a1eb8bf34b9bfc8e75f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 44355 } } peers { permanent_uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33739 } } peers { permanent_uuid: "a87ef90c67ba4512bbab31bd798bc6c4" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45475 } }
I20260806 19:35:09.384055 15124 raft_consensus.cc:2454] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab8ac6f1fe6449a9b329b1345f7a94bb in term 1.
I20260806 19:35:09.391316 14706 catalog_manager.cc:5495] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb reported cstate change: term changed from 0 to 1, leader changed from <none> to ab8ac6f1fe6449a9b329b1345f7a94bb (127.10.25.2). New cstate: current_term: 1 leader_uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0649cf5de3d4a1eb8bf34b9bfc8e75f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 44355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab8ac6f1fe6449a9b329b1345f7a94bb" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a87ef90c67ba4512bbab31bd798bc6c4" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45475 } health_report { overall_health: UNKNOWN } } }
I20260806 19:35:09.450143 15187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:09.443118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53620 (local address 127.10.25.2:33739)
0806 19:35:09.443397 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:09.443403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:09.443429 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:35:09.443566 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:09.443570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:09.443663 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:09.443851 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:35:09.444330 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.445177 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.446870 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.447337 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.447499 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:09.448084 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:35:09.448966 (+ 882us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:35:09.449276 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:09.449725 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 19:35:09.449820 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 19:35:09.462803 15124 raft_consensus.cc:1261] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4 [term 1 FOLLOWER]: Refusing update from remote peer ab8ac6f1fe6449a9b329b1345f7a94bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:35:09.462816 14853 raft_consensus.cc:1261] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f [term 1 FOLLOWER]: Refusing update from remote peer ab8ac6f1fe6449a9b329b1345f7a94bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:35:09.464170 15201 consensus_queue.cc:1035] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a87ef90c67ba4512bbab31bd798bc6c4" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:09.464887 15196 consensus_queue.cc:1035] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0649cf5de3d4a1eb8bf34b9bfc8e75f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 44355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:09.481894 15204 mvcc.cc:205] Tried to move back new op lower bound from 7315639949142745088 to 7315639948839952384. Current Snapshot: MvccSnapshot[applied={T|T < 7315639949142745088}]
I20260806 19:35:09.508868 14706 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:35:09.509389 14706 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:35:09.516420 14706 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18f7d022cffa4682ba302dc27be317b1: Sending DeleteTablet for 3 replicas of tablet fe020bcd9af5415da777ffab2a8d7547
I20260806 19:35:09.518303 15103 tablet_service.cc:1514] Processing DeleteTablet for tablet fe020bcd9af5415da777ffab2a8d7547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:09 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:48428
I20260806 19:35:09.518628 14967 tablet_service.cc:1514] Processing DeleteTablet for tablet fe020bcd9af5415da777ffab2a8d7547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:09 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:53612
I20260806 19:35:09.519852 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 14769
I20260806 19:35:09.519809 14832 tablet_service.cc:1514] Processing DeleteTablet for tablet fe020bcd9af5415da777ffab2a8d7547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:09 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:39512
I20260806 19:35:09.520041 15215 tablet_replica.cc:330] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb: stopping tablet replica
I20260806 19:35:09.520603 15215 raft_consensus.cc:2229] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:35:09.521394 15215 raft_consensus.cc:2258] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:09.522379 15214 tablet_replica.cc:330] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4: stopping tablet replica
I20260806 19:35:09.524026 15214 raft_consensus.cc:2229] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:09.525300 15214 raft_consensus.cc:2258] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:09.527923 15215 ts_tablet_manager.cc:1905] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:09.532130 15214 ts_tablet_manager.cc:1905] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:09.538583 15215 ts_tablet_manager.cc:1918] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:09.539111 15215 log.cc:1196] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/fe020bcd9af5415da777ffab2a8d7547
I20260806 19:35:09.540222 15215 ts_tablet_manager.cc:1939] T fe020bcd9af5415da777ffab2a8d7547 P ab8ac6f1fe6449a9b329b1345f7a94bb: Deleting consensus metadata
I20260806 19:35:09.542138 15214 ts_tablet_manager.cc:1918] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:09.542424 15214 log.cc:1196] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/fe020bcd9af5415da777ffab2a8d7547
I20260806 19:35:09.543008 14689 catalog_manager.cc:4841] TS ab8ac6f1fe6449a9b329b1345f7a94bb (127.10.25.2:33739): tablet fe020bcd9af5415da777ffab2a8d7547 (table test-table [id=10528d76d988499489726a3d9a34f297]) successfully deleted
I20260806 19:35:09.543707 15214 ts_tablet_manager.cc:1939] T fe020bcd9af5415da777ffab2a8d7547 P a87ef90c67ba4512bbab31bd798bc6c4: Deleting consensus metadata
I20260806 19:35:09.546281 14692 catalog_manager.cc:4841] TS a87ef90c67ba4512bbab31bd798bc6c4 (127.10.25.3:45475): tablet fe020bcd9af5415da777ffab2a8d7547 (table test-table [id=10528d76d988499489726a3d9a34f297]) successfully deleted
I20260806 19:35:09.550563 15216 tablet_replica.cc:330] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f: stopping tablet replica
I20260806 19:35:09.551628 15216 raft_consensus.cc:2229] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:09.552580 15216 raft_consensus.cc:2258] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:09.555938 15216 ts_tablet_manager.cc:1905] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 19:35:09 dist-test-slave-ss4n krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:35:09.570734 15186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:09.521655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39520 (local address 127.10.25.1:44355)
0806 19:35:09.521879 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:09.521886 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:09.529281 (+ 7395us) server_negotiation.cc:407] Connection header received
0806 19:35:09.543563 (+ 14282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:09.543571 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:09.543801 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:09.543951 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:09.546194 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.547891 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.551551 (+ 3660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.552399 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.552520 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:09.558939 (+ 6419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:09.558967 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:09.558993 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:09.559058 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:09.566813 (+ 7755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:09.567807 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:09.567817 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:09.567820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:09.568028 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:09.568528 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:09.568533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:09.568535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:09.569010 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:09.569394 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:09.569816 (+ 422us) server_negotiation.cc:299] Negotiation successful
0806 19:35:09.570110 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 19:35:09.570675 15216 ts_tablet_manager.cc:1918] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:09.571020 15216 log.cc:1196] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/fe020bcd9af5415da777ffab2a8d7547
I20260806 19:35:09.571802 14893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:09.722481 15216 ts_tablet_manager.cc:1939] T fe020bcd9af5415da777ffab2a8d7547 P c0649cf5de3d4a1eb8bf34b9bfc8e75f: Deleting consensus metadata
I20260806 19:35:09.725087 14692 catalog_manager.cc:4841] TS c0649cf5de3d4a1eb8bf34b9bfc8e75f (127.10.25.1:44355): tablet fe020bcd9af5415da777ffab2a8d7547 (table test-table [id=10528d76d988499489726a3d9a34f297]) successfully deleted
I20260806 19:35:09.725836 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 14769
I20260806 19:35:09.759212 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 14904
Aug 06 19:35:09 dist-test-slave-ss4n krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:35:09.785653 15187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:09.760333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53628 (local address 127.10.25.2:33739)
0806 19:35:09.760534 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:09.760541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:09.760570 (+ 29us) server_negotiation.cc:407] Connection header received
0806 19:35:09.760654 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:09.760660 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:09.760751 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:09.761043 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:09.764123 (+ 3080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.765733 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.768004 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.768737 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.768896 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:09.776203 (+ 7307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:09.776226 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:09.776243 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:09.776310 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:09.780225 (+ 3915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:09.781662 (+ 1437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:09.781672 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:09.781674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:09.781845 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:09.782467 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:09.782474 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:09.782477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:09.783204 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:09.783639 (+ 435us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:09.784138 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 19:35:09.784655 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 19:35:09.786877 15029 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:09.926311 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 14904
I20260806 19:35:09.953644 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 15041
Aug 06 19:35:09 dist-test-slave-ss4n krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:35:09.976315 15220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:09.955028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48438 (local address 127.10.25.3:45475)
0806 19:35:09.955710 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:09.955714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:09.955731 (+ 17us) server_negotiation.cc:407] Connection header received
0806 19:35:09.955791 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:09.955795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:09.955862 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:09.955993 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:09.957414 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.958378 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.959962 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:09.960423 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:09.960571 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:09.966136 (+ 5565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:09.966165 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:09.966183 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:09.966238 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:09.971633 (+ 5395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:09.973463 (+ 1830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:09.973475 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:09.973543 (+ 68us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:09.973793 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:09.974291 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:09.974297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:09.974301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:09.974776 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:09.975178 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:09.975640 (+ 462us) server_negotiation.cc:299] Negotiation successful
0806 19:35:09.975915 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":280,"threads_started":1}
I20260806 19:35:09.977054 15165 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:10.083314 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 15041
I20260806 19:35:10.104609 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 14674
I20260806 19:35:10.106086 14736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:10.227764 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 14674
2026-08-06T19:35:10Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (5573 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T19:35:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:35:10Z Disabled control of system clock
I20260806 19:35:10.318599 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:33265
--webserver_interface=127.10.25.62
--webserver_port=0
--builtin_ntp_servers=127.10.25.20:34583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:33265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 19:35:10.794297 15226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:10.794953 15226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:10.795068 15226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:10.806480 15226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:35:10.806646 15226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:10.806697 15226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:10.806762 15226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:35:10.806800 15226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:35:10.821847 15226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:34583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:33265
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:33265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:10.823662 15226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:10.825675 15226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:10.850498 15232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:10.854236 15234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:10.854431 15231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:10.855719 15226 server_base.cc:1034] running on GCE node
I20260806 19:35:10.857163 15226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:10.859599 15226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:10.863324 15226 hybrid_clock.cc:648] HybridClock initialized: now 1786044910863142 us; error 131 us; skew 500 ppm
I20260806 19:35:10.863971 15226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:35:10.870000 15226 webserver.cc:460] Webserver started at http://127.10.25.62:35263/ using document root <none> and password file <none>
I20260806 19:35:10.871325 15226 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:10.872254 15226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:10.872823 15226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:10.878326 15226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "5f4b47ef9c0d478ba0a197ad065afeb0"
format_stamp: "Formatted at 2026-08-06 19:35:10 on dist-test-slave-ss4n"
server_key: "ca987765bc51b175bcea57fcc4e3d636"
server_key_iv: "90e7457bc8e9cc70dadc342a098ded44"
server_key_version: "encryptionkey@0"
I20260806 19:35:10.880378 15226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "5f4b47ef9c0d478ba0a197ad065afeb0"
format_stamp: "Formatted at 2026-08-06 19:35:10 on dist-test-slave-ss4n"
server_key: "ca987765bc51b175bcea57fcc4e3d636"
server_key_iv: "90e7457bc8e9cc70dadc342a098ded44"
server_key_version: "encryptionkey@0"
I20260806 19:35:10.890071 15226 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.011s sys 0.000s
I20260806 19:35:10.900051 15240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:10.903049 15226 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 19:35:10.903488 15226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "5f4b47ef9c0d478ba0a197ad065afeb0"
format_stamp: "Formatted at 2026-08-06 19:35:10 on dist-test-slave-ss4n"
server_key: "ca987765bc51b175bcea57fcc4e3d636"
server_key_iv: "90e7457bc8e9cc70dadc342a098ded44"
server_key_version: "encryptionkey@0"
I20260806 19:35:10.904152 15226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:10.914318 15226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:10.915478 15226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:10.915822 15226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:10.946800 15226 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:33265
I20260806 19:35:10.946844 15293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:33265 every 8 connection(s)
I20260806 19:35:10.948485 15226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:35:10.949321 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 15226
I20260806 19:35:10.949617 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:35:10.952638 10340 external_mini_cluster.cc:1428] Setting key e0b25d4f967b9b5f96c07dd6eec9fc1c
I20260806 19:35:10.959607 15295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:10.972489 15298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:10.959383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45416 (local address 127.10.25.62:33265)
0806 19:35:10.962483 (+ 3100us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:10.962498 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:10.962551 (+ 53us) server_negotiation.cc:407] Connection header received
0806 19:35:10.964899 (+ 2348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:10.964944 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:10.965746 (+ 802us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:10.966209 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:10.968201 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:10.968209 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:10.968231 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:10.968251 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:10.968357 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:10.968378 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:10.968642 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:10.968787 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 19:35:10.969274 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2365,"thread_start_us":1679,"threads_started":1}
I20260806 19:35:10.984400 15295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0: Bootstrap starting.
I20260806 19:35:10.988781 15295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:10.990276 15295 log.cc:824] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:11.002413 15295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0: No bootstrap required, opened a new log
I20260806 19:35:11.020634 15295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4b47ef9c0d478ba0a197ad065afeb0" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 33265 } }
I20260806 19:35:11.021464 15295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:11.021727 15295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f4b47ef9c0d478ba0a197ad065afeb0, State: Initialized, Role: FOLLOWER
I20260806 19:35:11.022614 15295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4b47ef9c0d478ba0a197ad065afeb0" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 33265 } }
I20260806 19:35:11.022907 15295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:35:11.023077 15295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:35:11.023303 15295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:11.026288 15295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4b47ef9c0d478ba0a197ad065afeb0" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 33265 } }
I20260806 19:35:11.027637 15295 leader_election.cc:302] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f4b47ef9c0d478ba0a197ad065afeb0; no voters:
I20260806 19:35:11.030215 15295 leader_election.cc:288] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:35:11.030294 15300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:11.042986 15300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [term 1 LEADER]: Becoming Leader. State: Replica: 5f4b47ef9c0d478ba0a197ad065afeb0, State: Running, Role: LEADER
I20260806 19:35:11.044174 15300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4b47ef9c0d478ba0a197ad065afeb0" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 33265 } }
I20260806 19:35:11.046016 15295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:35:11.047998 15302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f4b47ef9c0d478ba0a197ad065afeb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4b47ef9c0d478ba0a197ad065afeb0" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 33265 } } }
I20260806 19:35:11.048452 15302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:11.047818 15301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f4b47ef9c0d478ba0a197ad065afeb0. Latest consensus state: current_term: 1 leader_uuid: "5f4b47ef9c0d478ba0a197ad065afeb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4b47ef9c0d478ba0a197ad065afeb0" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 33265 } } }
I20260806 19:35:11.049520 15301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:11.053894 15306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:35:11.065938 15306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:35:11.084291 15306 catalog_manager.cc:1347] Generated new cluster ID: c726bb34e0bf4025bd9d368afc7c68c5
I20260806 19:35:11.084568 15306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:35:11.123214 15306 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:35:11.123780 15306 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:35:11.138692 15306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0: Generated new TSK 0
I20260806 19:35:11.140895 15306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:35:11.160239 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:33265
--builtin_ntp_servers=127.10.25.20:34583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 19:35:11.711831 15319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:11.712183 15319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:11.712296 15319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:11.725184 15319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:11.725363 15319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:11.725623 15319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:35:11.739941 15319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:34583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:33265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:11.741681 15319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:11.744330 15319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:11.766932 15328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:11.766865 15324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:11.767059 15325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:11.769346 15319 server_base.cc:1034] running on GCE node
I20260806 19:35:11.770290 15319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:11.771581 15319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:11.772933 15319 hybrid_clock.cc:648] HybridClock initialized: now 1786044911772846 us; error 68 us; skew 500 ppm
I20260806 19:35:11.773458 15319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:35:11.778924 15319 webserver.cc:460] Webserver started at http://127.10.25.1:46211/ using document root <none> and password file <none>
I20260806 19:35:11.779877 15319 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:11.780063 15319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:11.780659 15319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:11.785017 15319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "95a2a2563bb94a8babf9dc16b49cce8b"
format_stamp: "Formatted at 2026-08-06 19:35:11 on dist-test-slave-ss4n"
server_key: "5269e105245f41f88eed2370d5e0501e"
server_key_iv: "310ebadf8cbb6a3ce7e553ce180a043a"
server_key_version: "encryptionkey@0"
I20260806 19:35:11.786481 15319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "95a2a2563bb94a8babf9dc16b49cce8b"
format_stamp: "Formatted at 2026-08-06 19:35:11 on dist-test-slave-ss4n"
server_key: "5269e105245f41f88eed2370d5e0501e"
server_key_iv: "310ebadf8cbb6a3ce7e553ce180a043a"
server_key_version: "encryptionkey@0"
I20260806 19:35:11.794936 15319 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260806 19:35:11.801780 15334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:11.804476 15319 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260806 19:35:11.805064 15319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "95a2a2563bb94a8babf9dc16b49cce8b"
format_stamp: "Formatted at 2026-08-06 19:35:11 on dist-test-slave-ss4n"
server_key: "5269e105245f41f88eed2370d5e0501e"
server_key_iv: "310ebadf8cbb6a3ce7e553ce180a043a"
server_key_version: "encryptionkey@0"
I20260806 19:35:11.805423 15319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:11.816510 15319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:11.817827 15319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:11.818533 15319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:11.820446 15319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:11.822737 15319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:11.822909 15319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:11.823091 15319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:11.823199 15319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:11.883656 15319 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:37515
I20260806 19:35:11.883692 15448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:37515 every 8 connection(s)
I20260806 19:35:11.885175 15319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:35:11.894289 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 15319
I20260806 19:35:11.894666 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:35:11.895561 10340 external_mini_cluster.cc:1428] Setting key 7843cb2f0e756bd2a4c7095affca7a34
I20260806 19:35:11.898373 15451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:11.888301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:42561 (local address 127.10.25.62:33265)
0806 19:35:11.889383 (+ 1082us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:11.889392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:11.891711 (+ 2319us) server_negotiation.cc:407] Connection header received
0806 19:35:11.894643 (+ 2932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:11.894655 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:11.894753 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:11.894936 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:11.897107 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:11.897112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:11.897116 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:11.897135 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:11.897207 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:11.897220 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:11.897393 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:11.897862 (+ 469us) server_negotiation.cc:299] Negotiation successful
0806 19:35:11.897945 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":880,"thread_start_us":660,"threads_started":1}
I20260806 19:35:11.899441 15452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:11.889599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:33265 (local address 127.10.25.1:42561)
0806 19:35:11.891368 (+ 1769us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:11.891453 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:11.893714 (+ 2261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:11.895254 (+ 1540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:11.895266 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:11.895397 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:11.896427 (+ 1030us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:35:11.896430 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:11.896492 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:35:11.896737 (+ 245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:11.897357 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:11.897371 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:11.897381 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 19:35:11.897969 (+ 588us) client_negotiation.cc:240] Negotiation successful
0806 19:35:11.898331 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"thread_start_us":526,"threads_started":1}
I20260806 19:35:11.901259 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:33265
--builtin_ntp_servers=127.10.25.20:34583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 19:35:11.901881 15449 heartbeater.cc:347] Connected to a master server at 127.10.25.62:33265
I20260806 19:35:11.903360 15449 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:11.904332 15449 heartbeater.cc:511] Master 127.10.25.62:33265 requested a full tablet report, sending...
I20260806 19:35:11.907595 15257 ts_manager.cc:194] Registered new tserver with Master: 95a2a2563bb94a8babf9dc16b49cce8b (127.10.25.1:37515)
W20260806 19:35:12.357147 15453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:12.357594 15453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:12.357784 15453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:12.367038 15453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:12.367740 15453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:12.368027 15453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:35:12.381165 15453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:34583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:33265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:12.382799 15453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:12.384944 15453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:12.407375 15458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:12.407672 15461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:12.408280 15453 server_base.cc:1034] running on GCE node
W20260806 19:35:12.408296 15459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:12.410501 15453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:12.413061 15453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:12.414733 15453 hybrid_clock.cc:648] HybridClock initialized: now 1786044912414574 us; error 113 us; skew 500 ppm
I20260806 19:35:12.415508 15453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:35:12.418627 15453 webserver.cc:460] Webserver started at http://127.10.25.2:37373/ using document root <none> and password file <none>
I20260806 19:35:12.419513 15453 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:12.419689 15453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:12.420087 15453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:12.423792 15453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "fe9bcbeea6aa42c5a6c039886c1ba3cf"
format_stamp: "Formatted at 2026-08-06 19:35:12 on dist-test-slave-ss4n"
server_key: "a64887924a7c81102883183397207629"
server_key_iv: "3898540b0b9671ed77c81feb6a6b3d70"
server_key_version: "encryptionkey@0"
I20260806 19:35:12.424923 15453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "fe9bcbeea6aa42c5a6c039886c1ba3cf"
format_stamp: "Formatted at 2026-08-06 19:35:12 on dist-test-slave-ss4n"
server_key: "a64887924a7c81102883183397207629"
server_key_iv: "3898540b0b9671ed77c81feb6a6b3d70"
server_key_version: "encryptionkey@0"
I20260806 19:35:12.431993 15453 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260806 19:35:12.439376 15467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:12.443174 15453 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.006s sys 0.000s
I20260806 19:35:12.443513 15453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "fe9bcbeea6aa42c5a6c039886c1ba3cf"
format_stamp: "Formatted at 2026-08-06 19:35:12 on dist-test-slave-ss4n"
server_key: "a64887924a7c81102883183397207629"
server_key_iv: "3898540b0b9671ed77c81feb6a6b3d70"
server_key_version: "encryptionkey@0"
I20260806 19:35:12.444232 15453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:12.452601 15453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:12.455317 15453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:12.456195 15453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:12.458577 15453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:12.461392 15453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:12.461642 15453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 19:35:12.461822 15453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:12.461930 15453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:12.521775 15453 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:43383
I20260806 19:35:12.522130 15584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:43383 every 8 connection(s)
I20260806 19:35:12.523517 15453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:35:12.531356 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 15453
I20260806 19:35:12.531842 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:35:12.532749 10340 external_mini_cluster.cc:1428] Setting key 8c62adb86056ab3a02a93219bd0a5c03
I20260806 19:35:12.537770 15587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:12.527119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:54517 (local address 127.10.25.62:33265)
0806 19:35:12.527749 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:12.527755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:12.529008 (+ 1253us) server_negotiation.cc:407] Connection header received
0806 19:35:12.533076 (+ 4068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:12.533082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:12.533152 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:12.533288 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:12.536215 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:12.536247 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:12.536256 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:12.536304 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:12.536646 (+ 342us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:12.536687 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:12.536847 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:12.537315 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 19:35:12.537382 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":358,"threads_started":1}
I20260806 19:35:12.537880 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:33265
--builtin_ntp_servers=127.10.25.20:34583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 19:35:12.539361 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:12.527619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:33265 (local address 127.10.25.2:54517)
0806 19:35:12.528780 (+ 1161us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:12.528832 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:12.532331 (+ 3499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:12.533796 (+ 1465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:12.533810 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:12.533972 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:12.535059 (+ 1087us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:35:12.535063 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:12.535148 (+ 85us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:35:12.535330 (+ 182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:12.536914 (+ 1584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:12.536941 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:12.536954 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 19:35:12.537402 (+ 448us) client_negotiation.cc:240] Negotiation successful
0806 19:35:12.537821 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":352,"threads_started":1}
I20260806 19:35:12.541445 15585 heartbeater.cc:347] Connected to a master server at 127.10.25.62:33265
I20260806 19:35:12.541903 15585 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:12.542615 15585 heartbeater.cc:511] Master 127.10.25.62:33265 requested a full tablet report, sending...
I20260806 19:35:12.544775 15257 ts_manager.cc:194] Registered new tserver with Master: fe9bcbeea6aa42c5a6c039886c1ba3cf (127.10.25.2:43383)
I20260806 19:35:12.912428 15449 heartbeater.cc:503] Master 127.10.25.62:33265 was elected leader, sending a full tablet report...
W20260806 19:35:12.975338 15589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:12.975736 15589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:12.975922 15589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:12.985566 15589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:12.985785 15589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:12.985968 15589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:35:13.001400 15589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:34583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:33265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:13.003754 15589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:13.006187 15589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:13.026858 15595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:13.027063 15594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:13.027643 15597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:13.029178 15589 server_base.cc:1034] running on GCE node
I20260806 19:35:13.030167 15589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:13.031657 15589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:13.035257 15589 hybrid_clock.cc:648] HybridClock initialized: now 1786044913035147 us; error 83 us; skew 500 ppm
I20260806 19:35:13.035758 15589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:35:13.041208 15589 webserver.cc:460] Webserver started at http://127.10.25.3:39351/ using document root <none> and password file <none>
I20260806 19:35:13.042629 15589 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:13.042822 15589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:13.043653 15589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:13.048342 15589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "2e41427dfe93401aa0dadbc1a7f89d38"
format_stamp: "Formatted at 2026-08-06 19:35:13 on dist-test-slave-ss4n"
server_key: "e65d0204640c09378fcc136a5a938883"
server_key_iv: "d55e9b51bc640b5a333e465d788e12a6"
server_key_version: "encryptionkey@0"
I20260806 19:35:13.049942 15589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "2e41427dfe93401aa0dadbc1a7f89d38"
format_stamp: "Formatted at 2026-08-06 19:35:13 on dist-test-slave-ss4n"
server_key: "e65d0204640c09378fcc136a5a938883"
server_key_iv: "d55e9b51bc640b5a333e465d788e12a6"
server_key_version: "encryptionkey@0"
I20260806 19:35:13.060284 15589 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.005s sys 0.004s
I20260806 19:35:13.065313 15603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:13.067574 15589 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:35:13.067847 15589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "2e41427dfe93401aa0dadbc1a7f89d38"
format_stamp: "Formatted at 2026-08-06 19:35:13 on dist-test-slave-ss4n"
server_key: "e65d0204640c09378fcc136a5a938883"
server_key_iv: "d55e9b51bc640b5a333e465d788e12a6"
server_key_version: "encryptionkey@0"
I20260806 19:35:13.068142 15589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:13.077085 15589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:13.078251 15589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:13.078586 15589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:13.080338 15589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:13.082710 15589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:13.082881 15589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:13.083066 15589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:13.083177 15589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:13.146366 15589 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:41153
I20260806 19:35:13.146502 15719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:41153 every 8 connection(s)
I20260806 19:35:13.148701 15589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:35:13.154350 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 15589
I20260806 19:35:13.154731 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:35:13.155325 10340 external_mini_cluster.cc:1428] Setting key cc77282e4e26231da5e6394070b9a2a9
I20260806 19:35:13.165100 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.152904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:49869 (local address 127.10.25.62:33265)
0806 19:35:13.154029 (+ 1125us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:13.154037 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:13.158009 (+ 3972us) server_negotiation.cc:407] Connection header received
0806 19:35:13.160409 (+ 2400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:13.160414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:13.160484 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:13.160661 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:13.163830 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:13.163837 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:13.163841 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:13.163864 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:13.163922 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:13.163937 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:13.164073 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:13.164742 (+ 669us) server_negotiation.cc:299] Negotiation successful
0806 19:35:13.164810 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":974,"thread_start_us":478,"threads_started":1}
I20260806 19:35:13.166671 15723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.153298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:33265 (local address 127.10.25.3:49869)
0806 19:35:13.154608 (+ 1310us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:13.154670 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:13.159611 (+ 4941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:13.161473 (+ 1862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:13.161544 (+ 71us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:13.161714 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:13.163117 (+ 1403us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:35:13.163120 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:13.163190 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:35:13.163528 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:13.164158 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:13.164168 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:13.164175 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 19:35:13.164727 (+ 552us) client_negotiation.cc:240] Negotiation successful
0806 19:35:13.165007 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":396,"threads_started":1}
I20260806 19:35:13.169901 15720 heartbeater.cc:347] Connected to a master server at 127.10.25.62:33265
I20260806 19:35:13.170313 15720 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:13.171017 15720 heartbeater.cc:511] Master 127.10.25.62:33265 requested a full tablet report, sending...
I20260806 19:35:13.174150 15257 ts_manager.cc:194] Registered new tserver with Master: 2e41427dfe93401aa0dadbc1a7f89d38 (127.10.25.3:41153)
I20260806 19:35:13.175959 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:35:13.185577 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.182434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53336 (local address 127.10.25.62:33265)
0806 19:35:13.182761 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:13.182770 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:13.183146 (+ 376us) server_negotiation.cc:407] Connection header received
0806 19:35:13.183381 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:13.183386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:13.183443 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:13.183568 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:13.184813 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:13.184819 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:13.184822 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:13.184841 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:13.184894 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:13.184908 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:13.185124 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:13.185180 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 19:35:13.185245 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260806 19:35:13.193282 15257 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53336:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:35:13.202310 15257 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:35:13.237174 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.221926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:41153 (local address 127.0.0.1:40418)
0806 19:35:13.231137 (+ 9211us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:13.231161 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:13.231904 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:13.233846 (+ 1942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:13.233856 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:13.233974 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:13.235173 (+ 1199us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:35:13.235179 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:13.235221 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:35:13.235314 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:13.236096 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:13.236105 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:13.236113 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 19:35:13.236409 (+ 296us) client_negotiation.cc:240] Negotiation successful
0806 19:35:13.236522 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8911,"thread_start_us":2393,"threads_started":1}
I20260806 19:35:13.238003 15735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.221544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:43383 (local address 127.0.0.1:55282)
0806 19:35:13.229458 (+ 7914us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:13.229655 (+ 197us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:13.229931 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:13.233846 (+ 3915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:13.233856 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:13.233975 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:13.235346 (+ 1371us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:35:13.235349 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:13.235363 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:35:13.235441 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:13.236032 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:13.236058 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:13.236074 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 19:35:13.236661 (+ 587us) client_negotiation.cc:240] Negotiation successful
0806 19:35:13.236732 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3520,"thread_start_us":8784,"threads_started":1}
I20260806 19:35:13.238693 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.222914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40418 (local address 127.10.25.3:41153)
0806 19:35:13.224021 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:13.224066 (+ 45us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:13.231347 (+ 7281us) server_negotiation.cc:407] Connection header received
0806 19:35:13.232261 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:13.232269 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:13.232438 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:13.232575 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:13.235511 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:13.235525 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:13.235545 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:13.235583 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:13.235648 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:13.235669 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:13.238059 (+ 2390us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:13.238158 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 19:35:13.238310 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":826,"thread_start_us":559,"threads_started":1}
I20260806 19:35:13.238828 15733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.221428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55282 (local address 127.10.25.2:43383)
0806 19:35:13.231869 (+ 10441us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:13.231879 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:13.231909 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:35:13.232104 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:13.232123 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:13.232426 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:13.232663 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:13.235632 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:13.235641 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:13.235654 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:13.235686 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:13.235747 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:13.235766 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:13.235926 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:13.238442 (+ 2516us) server_negotiation.cc:299] Negotiation successful
0806 19:35:13.238540 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10269,"thread_start_us":10076,"threads_started":1}
I20260806 19:35:13.240983 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.228301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57520 (local address 127.10.25.1:37515)
0806 19:35:13.234401 (+ 6100us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:13.234431 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:13.235445 (+ 1014us) server_negotiation.cc:407] Connection header received
0806 19:35:13.235560 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:13.235569 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:13.235831 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:13.236022 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:13.239304 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:13.239314 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:13.239331 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:13.239354 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:13.239438 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:13.239460 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:13.239664 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:13.239853 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 19:35:13.240337 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5757,"thread_start_us":5572,"threads_started":1}
I20260806 19:35:13.241259 15737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.227202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:37515 (local address 127.0.0.1:57520)
0806 19:35:13.235109 (+ 7907us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:13.235129 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:13.235272 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:13.237213 (+ 1941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:13.237218 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:13.237248 (+ 30us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:13.239010 (+ 1762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:35:13.239014 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:13.239035 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:35:13.239095 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:13.239615 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:13.239621 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:13.239626 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 19:35:13.239782 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 19:35:13.239841 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7696,"thread_start_us":7127,"threads_started":1}
I20260806 19:35:13.243494 15652 tablet_service.cc:1467] Processing CreateTablet for tablet 2e98043394f54dd8b0f7449d1ac07c29 (DEFAULT_TABLE table=test-table [id=e23b3a2d35e94b9f9c2a84a9165a80ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:13.243492 15518 tablet_service.cc:1467] Processing CreateTablet for tablet 2e98043394f54dd8b0f7449d1ac07c29 (DEFAULT_TABLE table=test-table [id=e23b3a2d35e94b9f9c2a84a9165a80ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:13.245639 15518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e98043394f54dd8b0f7449d1ac07c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:13.245639 15652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e98043394f54dd8b0f7449d1ac07c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:13.249320 15383 tablet_service.cc:1467] Processing CreateTablet for tablet 2e98043394f54dd8b0f7449d1ac07c29 (DEFAULT_TABLE table=test-table [id=e23b3a2d35e94b9f9c2a84a9165a80ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:13.252341 15383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e98043394f54dd8b0f7449d1ac07c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:13.268318 15740 tablet_bootstrap.cc:492] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf: Bootstrap starting.
I20260806 19:35:13.269594 15739 tablet_bootstrap.cc:492] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38: Bootstrap starting.
I20260806 19:35:13.270895 15741 tablet_bootstrap.cc:492] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b: Bootstrap starting.
I20260806 19:35:13.274969 15741 tablet_bootstrap.cc:654] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:13.275496 15740 tablet_bootstrap.cc:654] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:13.276345 15741 log.cc:824] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:13.277346 15739 tablet_bootstrap.cc:654] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:13.278137 15740 log.cc:824] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:13.280469 15741 tablet_bootstrap.cc:492] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b: No bootstrap required, opened a new log
I20260806 19:35:13.280827 15741 ts_tablet_manager.cc:1397] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 19:35:13.280826 15739 log.cc:824] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:13.282740 15740 tablet_bootstrap.cc:492] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf: No bootstrap required, opened a new log
I20260806 19:35:13.283069 15740 ts_tablet_manager.cc:1397] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.005s
I20260806 19:35:13.284682 15739 tablet_bootstrap.cc:492] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38: No bootstrap required, opened a new log
I20260806 19:35:13.284979 15739 ts_tablet_manager.cc:1397] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.000s
I20260806 19:35:13.291934 15741 raft_consensus.cc:348] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe9bcbeea6aa42c5a6c039886c1ba3cf" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 43383 } } peers { permanent_uuid: "2e41427dfe93401aa0dadbc1a7f89d38" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41153 } } peers { permanent_uuid: "95a2a2563bb94a8babf9dc16b49cce8b" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37515 } }
I20260806 19:35:13.292502 15741 raft_consensus.cc:374] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:13.292614 15741 raft_consensus.cc:729] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95a2a2563bb94a8babf9dc16b49cce8b, State: Initialized, Role: FOLLOWER
I20260806 19:35:13.293444 15741 consensus_queue.cc:260] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe9bcbeea6aa42c5a6c039886c1ba3cf" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 43383 } } peers { permanent_uuid: "2e41427dfe93401aa0dadbc1a7f89d38" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41153 } } peers { permanent_uuid: "95a2a2563bb94a8babf9dc16b49cce8b" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37515 } }
I20260806 19:35:13.295415 15741 ts_tablet_manager.cc:1428] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260806 19:35:13.297992 15739 raft_consensus.cc:348] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe9bcbeea6aa42c5a6c039886c1ba3cf" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 43383 } } peers { permanent_uuid: "2e41427dfe93401aa0dadbc1a7f89d38" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41153 } } peers { permanent_uuid: "95a2a2563bb94a8babf9dc16b49cce8b" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37515 } }
I20260806 19:35:13.297941 15740 raft_consensus.cc:348] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe9bcbeea6aa42c5a6c039886c1ba3cf" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 43383 } } peers { permanent_uuid: "2e41427dfe93401aa0dadbc1a7f89d38" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41153 } } peers { permanent_uuid: "95a2a2563bb94a8babf9dc16b49cce8b" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37515 } }
I20260806 19:35:13.298444 15739 raft_consensus.cc:374] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:13.298475 15740 raft_consensus.cc:374] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:13.298580 15739 raft_consensus.cc:729] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e41427dfe93401aa0dadbc1a7f89d38, State: Initialized, Role: FOLLOWER
I20260806 19:35:13.298578 15740 raft_consensus.cc:729] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe9bcbeea6aa42c5a6c039886c1ba3cf, State: Initialized, Role: FOLLOWER
I20260806 19:35:13.299437 15739 consensus_queue.cc:260] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe9bcbeea6aa42c5a6c039886c1ba3cf" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 43383 } } peers { permanent_uuid: "2e41427dfe93401aa0dadbc1a7f89d38" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41153 } } peers { permanent_uuid: "95a2a2563bb94a8babf9dc16b49cce8b" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37515 } }
I20260806 19:35:13.299398 15740 consensus_queue.cc:260] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe9bcbeea6aa42c5a6c039886c1ba3cf" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 43383 } } peers { permanent_uuid: "2e41427dfe93401aa0dadbc1a7f89d38" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41153 } } peers { permanent_uuid: "95a2a2563bb94a8babf9dc16b49cce8b" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37515 } }
I20260806 19:35:13.300727 15745 raft_consensus.cc:482] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:35:13.301618 15740 ts_tablet_manager.cc:1428] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf: Time spent starting tablet: real 0.018s user 0.019s sys 0.000s
I20260806 19:35:13.301318 15745 raft_consensus.cc:504] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe9bcbeea6aa42c5a6c039886c1ba3cf" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 43383 } } peers { permanent_uuid: "2e41427dfe93401aa0dadbc1a7f89d38" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41153 } } peers { permanent_uuid: "95a2a2563bb94a8babf9dc16b49cce8b" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37515 } }
I20260806 19:35:13.302129 15585 heartbeater.cc:503] Master 127.10.25.62:33265 was elected leader, sending a full tablet report...
I20260806 19:35:13.302793 15720 heartbeater.cc:503] Master 127.10.25.62:33265 was elected leader, sending a full tablet report...
I20260806 19:35:13.303594 15739 ts_tablet_manager.cc:1428] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38: Time spent starting tablet: real 0.018s user 0.016s sys 0.003s
I20260806 19:35:13.306277 15745 leader_election.cc:288] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe9bcbeea6aa42c5a6c039886c1ba3cf (127.10.25.2:43383), 2e41427dfe93401aa0dadbc1a7f89d38 (127.10.25.3:41153)
I20260806 19:35:13.312251 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.306505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:54051 (local address 127.10.25.3:41153)
0806 19:35:13.306721 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:13.306726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:13.306747 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:35:13.307051 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:13.307062 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:13.307157 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:13.307250 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:13.311020 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:13.311027 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:13.311032 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:13.311065 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:13.311125 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:13.311144 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:13.311300 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:13.311888 (+ 588us) server_negotiation.cc:299] Negotiation successful
0806 19:35:13.311980 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 19:35:13.313066 15748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.305525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:41153 (local address 127.10.25.1:54051)
0806 19:35:13.306536 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:13.306566 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:13.306772 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:13.307554 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:13.307561 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:13.307593 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:13.310509 (+ 2916us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:35:13.310516 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:13.310554 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:35:13.310723 (+ 169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:13.311375 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:13.311383 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:13.311400 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 19:35:13.312596 (+ 1196us) client_negotiation.cc:240] Negotiation successful
0806 19:35:13.312665 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"thread_start_us":447,"threads_started":1}
I20260806 19:35:13.314371 15673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e98043394f54dd8b0f7449d1ac07c29" candidate_uuid: "95a2a2563bb94a8babf9dc16b49cce8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e41427dfe93401aa0dadbc1a7f89d38" is_pre_election: true
I20260806 19:35:13.314941 15673 raft_consensus.cc:2454] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95a2a2563bb94a8babf9dc16b49cce8b in term 0.
I20260806 19:35:13.315831 15335 leader_election.cc:302] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e41427dfe93401aa0dadbc1a7f89d38, 95a2a2563bb94a8babf9dc16b49cce8b; no voters:
I20260806 19:35:13.316501 15733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.306159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:44511 (local address 127.10.25.2:43383)
0806 19:35:13.306527 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:13.306536 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:13.311741 (+ 5205us) server_negotiation.cc:407] Connection header received
0806 19:35:13.311987 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:13.311994 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:13.312099 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:13.312179 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:13.315494 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:13.315503 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:13.315509 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:13.315607 (+ 98us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:13.315674 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:13.315691 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:13.315808 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:13.316108 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 19:35:13.316182 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215}
I20260806 19:35:13.316442 15745 raft_consensus.cc:2781] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:35:13.316934 15745 raft_consensus.cc:482] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:35:13.317199 15745 raft_consensus.cc:3037] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:13.317373 15749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.306699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:43383 (local address 127.10.25.1:44511)
0806 19:35:13.311608 (+ 4909us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:13.311632 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:13.311819 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:13.312413 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:13.312418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:13.312448 (+ 30us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:13.313448 (+ 1000us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:35:13.313454 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:13.313637 (+ 183us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:35:13.313762 (+ 125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:13.315853 (+ 2091us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:13.315862 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:13.315871 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 19:35:13.316083 (+ 212us) client_negotiation.cc:240] Negotiation successful
0806 19:35:13.316157 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4701,"thread_start_us":4420,"threads_started":1}
I20260806 19:35:13.318753 15538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e98043394f54dd8b0f7449d1ac07c29" candidate_uuid: "95a2a2563bb94a8babf9dc16b49cce8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe9bcbeea6aa42c5a6c039886c1ba3cf" is_pre_election: true
I20260806 19:35:13.319748 15538 raft_consensus.cc:2454] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95a2a2563bb94a8babf9dc16b49cce8b in term 0.
I20260806 19:35:13.321372 15745 raft_consensus.cc:504] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe9bcbeea6aa42c5a6c039886c1ba3cf" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 43383 } } peers { permanent_uuid: "2e41427dfe93401aa0dadbc1a7f89d38" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41153 } } peers { permanent_uuid: "95a2a2563bb94a8babf9dc16b49cce8b" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37515 } }
I20260806 19:35:13.322456 15745 leader_election.cc:288] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [CANDIDATE]: Term 1 election: Requested vote from peers fe9bcbeea6aa42c5a6c039886c1ba3cf (127.10.25.2:43383), 2e41427dfe93401aa0dadbc1a7f89d38 (127.10.25.3:41153)
I20260806 19:35:13.323150 15538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e98043394f54dd8b0f7449d1ac07c29" candidate_uuid: "95a2a2563bb94a8babf9dc16b49cce8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe9bcbeea6aa42c5a6c039886c1ba3cf"
I20260806 19:35:13.323236 15673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e98043394f54dd8b0f7449d1ac07c29" candidate_uuid: "95a2a2563bb94a8babf9dc16b49cce8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e41427dfe93401aa0dadbc1a7f89d38"
I20260806 19:35:13.323446 15673 raft_consensus.cc:3037] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:13.323424 15538 raft_consensus.cc:3037] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:13.328612 15538 raft_consensus.cc:2454] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95a2a2563bb94a8babf9dc16b49cce8b in term 1.
I20260806 19:35:13.328612 15673 raft_consensus.cc:2454] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95a2a2563bb94a8babf9dc16b49cce8b in term 1.
I20260806 19:35:13.329523 15335 leader_election.cc:302] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e41427dfe93401aa0dadbc1a7f89d38, 95a2a2563bb94a8babf9dc16b49cce8b; no voters:
I20260806 19:35:13.330044 15745 raft_consensus.cc:2781] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:13.330670 15745 raft_consensus.cc:686] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [term 1 LEADER]: Becoming Leader. State: Replica: 95a2a2563bb94a8babf9dc16b49cce8b, State: Running, Role: LEADER
I20260806 19:35:13.331192 15745 consensus_queue.cc:237] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe9bcbeea6aa42c5a6c039886c1ba3cf" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 43383 } } peers { permanent_uuid: "2e41427dfe93401aa0dadbc1a7f89d38" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41153 } } peers { permanent_uuid: "95a2a2563bb94a8babf9dc16b49cce8b" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37515 } }
I20260806 19:35:13.336779 15256 catalog_manager.cc:5495] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 95a2a2563bb94a8babf9dc16b49cce8b (127.10.25.1). New cstate: current_term: 1 leader_uuid: "95a2a2563bb94a8babf9dc16b49cce8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe9bcbeea6aa42c5a6c039886c1ba3cf" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 43383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e41427dfe93401aa0dadbc1a7f89d38" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95a2a2563bb94a8babf9dc16b49cce8b" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37515 } health_report { overall_health: HEALTHY } } }
I20260806 19:35:13.367899 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.363931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57524 (local address 127.10.25.1:37515)
0806 19:35:13.364115 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:13.364120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:13.364140 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:35:13.364452 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:13.364457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:13.364548 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:13.364695 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:13.366604 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:13.366612 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:13.366617 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:13.366652 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:13.366718 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:13.366737 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:13.366991 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:13.367600 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 19:35:13.367674 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 19:35:13.381278 15673 raft_consensus.cc:1261] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38 [term 1 FOLLOWER]: Refusing update from remote peer 95a2a2563bb94a8babf9dc16b49cce8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:35:13.382205 15538 raft_consensus.cc:1261] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf [term 1 FOLLOWER]: Refusing update from remote peer 95a2a2563bb94a8babf9dc16b49cce8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:35:13.382771 15750 consensus_queue.cc:1035] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e41427dfe93401aa0dadbc1a7f89d38" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:13.383543 15745 consensus_queue.cc:1035] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe9bcbeea6aa42c5a6c039886c1ba3cf" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 43383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 19:35:13.388484 15450 tablet.cc:2369] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:35:13.399694 15753 mvcc.cc:205] Tried to move back new op lower bound from 7315639965197352960 to 7315639965008867328. Current Snapshot: MvccSnapshot[applied={T|T < 7315639965197352960}]
W20260806 19:35:13.401403 15721 tablet.cc:2369] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:35:13.433089 15256 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:35:13.434759 15256 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53336:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:35:13.440227 15256 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f4b47ef9c0d478ba0a197ad065afeb0: Sending DeleteTablet for 3 replicas of tablet 2e98043394f54dd8b0f7449d1ac07c29
I20260806 19:35:13.442368 15383 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e98043394f54dd8b0f7449d1ac07c29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:13 UTC) from {username='slave'} at 127.0.0.1:57520
I20260806 19:35:13.442231 15518 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e98043394f54dd8b0f7449d1ac07c29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:13 UTC) from {username='slave'} at 127.0.0.1:55282
I20260806 19:35:13.443441 15652 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e98043394f54dd8b0f7449d1ac07c29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:13 UTC) from {username='slave'} at 127.0.0.1:40418
I20260806 19:35:13.447062 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 15319
I20260806 19:35:13.456393 15766 tablet_replica.cc:330] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38: stopping tablet replica
I20260806 19:35:13.457702 15766 raft_consensus.cc:2229] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:13.458011 15765 tablet_replica.cc:330] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf: stopping tablet replica
I20260806 19:35:13.458338 15766 raft_consensus.cc:2258] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:13.458495 15765 raft_consensus.cc:2229] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:13.459532 15765 raft_consensus.cc:2258] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:13.461949 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.450617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57528 (local address 127.10.25.1:37515)
0806 19:35:13.451694 (+ 1077us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:13.451701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:13.455553 (+ 3852us) server_negotiation.cc:407] Connection header received
0806 19:35:13.455982 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:13.455989 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:13.456091 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:13.456204 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:13.460881 (+ 4677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:13.460892 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:13.460899 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:13.460933 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:13.460989 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:13.461012 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:13.461154 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:13.461521 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 19:35:13.461604 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":885}
I20260806 19:35:13.462560 15765 ts_tablet_manager.cc:1905] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:13.463114 15766 ts_tablet_manager.cc:1905] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:13.463380 15444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:13.478250 15766 ts_tablet_manager.cc:1918] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:13.478489 15766 log.cc:1196] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/2e98043394f54dd8b0f7449d1ac07c29
I20260806 19:35:13.478475 15765 ts_tablet_manager.cc:1918] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:13.479014 15765 log.cc:1196] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/2e98043394f54dd8b0f7449d1ac07c29
I20260806 19:35:13.479391 15766 ts_tablet_manager.cc:1939] T 2e98043394f54dd8b0f7449d1ac07c29 P 2e41427dfe93401aa0dadbc1a7f89d38: Deleting consensus metadata
I20260806 19:35:13.481119 15765 ts_tablet_manager.cc:1939] T 2e98043394f54dd8b0f7449d1ac07c29 P fe9bcbeea6aa42c5a6c039886c1ba3cf: Deleting consensus metadata
I20260806 19:35:13.483908 15244 catalog_manager.cc:4841] TS fe9bcbeea6aa42c5a6c039886c1ba3cf (127.10.25.2:43383): tablet 2e98043394f54dd8b0f7449d1ac07c29 (table test-table [id=e23b3a2d35e94b9f9c2a84a9165a80ce]) successfully deleted
I20260806 19:35:13.484881 15241 catalog_manager.cc:4841] TS 2e41427dfe93401aa0dadbc1a7f89d38 (127.10.25.3:41153): tablet 2e98043394f54dd8b0f7449d1ac07c29 (table test-table [id=e23b3a2d35e94b9f9c2a84a9165a80ce]) successfully deleted
I20260806 19:35:13.607890 15764 tablet_replica.cc:330] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b: stopping tablet replica
I20260806 19:35:13.608500 15764 raft_consensus.cc:2229] T 2e98043394f54dd8b0f7449d1ac07c29 P 95a2a2563bb94a8babf9dc16b49cce8b [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:35:13.608664 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 15319
I20260806 19:35:13.635497 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 15453
W20260806 19:35:13.635497 15241 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.25.1:37515 (error 108)
I20260806 19:35:13.639350 15737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.638352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:37515 (local address 127.0.0.1:57542)
0806 19:35:13.638907 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:13.639152 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260806 19:35:13.639675 15241 catalog_manager.cc:4584] TS 95a2a2563bb94a8babf9dc16b49cce8b (127.10.25.1:37515): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e98043394f54dd8b0f7449d1ac07c29: Network error: Client connection negotiation failed: client connection to 127.10.25.1:37515: connect: Connection refused (error 111)
I20260806 19:35:13.640985 15733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.636939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55284 (local address 127.10.25.2:43383)
0806 19:35:13.637222 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:13.637229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:13.637264 (+ 35us) server_negotiation.cc:407] Connection header received
0806 19:35:13.637733 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:13.637742 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:13.637914 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:13.638066 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:13.640114 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:13.640122 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:13.640126 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:13.640157 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:13.640206 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:13.640225 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:13.640371 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:13.640632 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 19:35:13.640715 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260806 19:35:13.641999 15579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:13.693833 15737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.693288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:37515 (local address 127.0.0.1:57548)
0806 19:35:13.693569 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:13.693661 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260806 19:35:13.695019 15737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.694630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:37515 (local address 127.0.0.1:57556)
0806 19:35:13.694802 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:13.694887 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 19:35:13.756197 15737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.755697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:37515 (local address 127.0.0.1:57564)
0806 19:35:13.755930 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:13.756025 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260806 19:35:13.759639 15737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.758952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:37515 (local address 127.0.0.1:57566)
0806 19:35:13.759296 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:13.759427 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
I20260806 19:35:13.760429 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 15453
I20260806 19:35:13.784809 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 15589
I20260806 19:35:13.795104 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.786230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40430 (local address 127.10.25.3:41153)
0806 19:35:13.786431 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:13.786435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:13.786771 (+ 336us) server_negotiation.cc:407] Connection header received
0806 19:35:13.787175 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:13.787302 (+ 127us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:13.787419 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:13.787615 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:13.794090 (+ 6475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:13.794097 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:13.794103 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:13.794129 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:13.794173 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:13.794190 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:13.794343 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:13.794626 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 19:35:13.794733 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 19:35:13.796209 15715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:13.842502 15737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.840914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:37515 (local address 127.0.0.1:57582)
0806 19:35:13.841890 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:13.842042 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433}
I20260806 19:35:13.844503 15737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:13.843514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:37515 (local address 127.0.0.1:57588)
0806 19:35:13.843990 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:13.844147 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:37515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333}
I20260806 19:35:13.909298 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 15589
I20260806 19:35:13.937592 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 15226
I20260806 19:35:13.939510 15289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:14.082592 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 15226
2026-08-06T19:35:14Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3851 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 19:35:14.131299 10340 mini_oidc.cc:152] Starting JWKS server
I20260806 19:35:14.137921 10340 webserver.cc:460] Webserver started at https://127.0.0.1:43537/ using document root <none> and password file <none>
I20260806 19:35:14.138738 10340 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 19:35:14.139649 10340 webserver.cc:460] Webserver started at http://127.0.0.1:36485/ using document root <none> and password file <none>
2026-08-06T19:35:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:35:14Z Disabled control of system clock
I20260806 19:35:14.184399 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:40851
--webserver_interface=127.10.25.62
--webserver_port=0
--builtin_ntp_servers=127.10.25.20:45567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:40851
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43537/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/testchainca.cert with env {}
W20260806 19:35:14.639482 15778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:14.639842 15778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:14.640017 15778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:14.650435 15778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:35:14.650604 15778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:14.650648 15778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:14.650799 15778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:35:14.650882 15778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 19:35:14.650991 15778 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 19:35:14.651162 15778 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43537/jwks.json
I20260806 19:35:14.663146 15778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:45567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:40851
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:40851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43537/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:14.664731 15778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:14.666342 15778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:14.683492 15783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:14.683424 15786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:14.683519 15784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:14.685141 15778 server_base.cc:1034] running on GCE node
I20260806 19:35:14.687665 15778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:14.689733 15778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:14.691054 15778 hybrid_clock.cc:648] HybridClock initialized: now 1786044914690989 us; error 85 us; skew 500 ppm
I20260806 19:35:14.691588 15778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:35:14.695549 15778 webserver.cc:460] Webserver started at http://127.10.25.62:40111/ using document root <none> and password file <none>
I20260806 19:35:14.696506 15778 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:14.697062 15778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:14.697464 15778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:14.700979 15778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "91a72ea647ed414da1325abff9174bad"
format_stamp: "Formatted at 2026-08-06 19:35:14 on dist-test-slave-ss4n"
server_key: "36da0aa994f0275eb89eaf0b1ab8891b"
server_key_iv: "a9587e08bfa51841d6ec29ce552b2f59"
server_key_version: "encryptionkey@0"
I20260806 19:35:14.702466 15778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "91a72ea647ed414da1325abff9174bad"
format_stamp: "Formatted at 2026-08-06 19:35:14 on dist-test-slave-ss4n"
server_key: "36da0aa994f0275eb89eaf0b1ab8891b"
server_key_iv: "a9587e08bfa51841d6ec29ce552b2f59"
server_key_version: "encryptionkey@0"
I20260806 19:35:14.711315 15778 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.005s
I20260806 19:35:14.718068 15792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.001s user 0.000s sys 0.000s
I20260806 19:35:14.720935 15778 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.001s
I20260806 19:35:14.721719 15778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "91a72ea647ed414da1325abff9174bad"
format_stamp: "Formatted at 2026-08-06 19:35:14 on dist-test-slave-ss4n"
server_key: "36da0aa994f0275eb89eaf0b1ab8891b"
server_key_iv: "a9587e08bfa51841d6ec29ce552b2f59"
server_key_version: "encryptionkey@0"
I20260806 19:35:14.722380 15778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:14.739960 15778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:14.740970 15778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:14.741343 15778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:14.770171 15778 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:40851
I20260806 19:35:14.770193 15846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:40851 every 8 connection(s)
I20260806 19:35:14.772394 15778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:35:14.779146 15847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:14.780267 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 15778
I20260806 19:35:14.780552 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:35:14.781380 10340 external_mini_cluster.cc:1428] Setting key 1cf02083beda0d7492b485213092a331
I20260806 19:35:14.798756 15847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad: Bootstrap starting.
I20260806 19:35:14.802225 15850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:14.784999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59306 (local address 127.10.25.62:40851)
0806 19:35:14.786340 (+ 1341us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:14.786354 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:14.786406 (+ 52us) server_negotiation.cc:407] Connection header received
0806 19:35:14.788340 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:14.788370 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:14.788995 (+ 625us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:14.789438 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:14.790536 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:14.792449 (+ 1913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:14.794900 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:14.796248 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:14.796511 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:14.799079 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:14.799092 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:14.799110 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:14.799742 (+ 632us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:14.799842 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:14.799869 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:14.800000 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:14.800531 (+ 531us) server_negotiation.cc:299] Negotiation successful
0806 19:35:14.800998 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":515,"threads_started":1}
I20260806 19:35:14.803018 15847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:14.804486 15847 log.cc:824] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:14.807862 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:35:14.808427 15847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad: No bootstrap required, opened a new log
I20260806 19:35:14.818468 15847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a72ea647ed414da1325abff9174bad" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40851 } }
I20260806 19:35:14.818946 15847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:14.819060 15847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91a72ea647ed414da1325abff9174bad, State: Initialized, Role: FOLLOWER
I20260806 19:35:14.819700 15847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a72ea647ed414da1325abff9174bad" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40851 } }
I20260806 19:35:14.819983 15847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:35:14.820140 15847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:35:14.820343 15847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:14.822870 15847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a72ea647ed414da1325abff9174bad" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40851 } }
I20260806 19:35:14.823518 15847 leader_election.cc:302] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91a72ea647ed414da1325abff9174bad; no voters:
I20260806 19:35:14.824450 15847 leader_election.cc:288] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:35:14.824543 15853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:14.827032 15853 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [term 1 LEADER]: Becoming Leader. State: Replica: 91a72ea647ed414da1325abff9174bad, State: Running, Role: LEADER
I20260806 19:35:14.827934 15847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:35:14.828091 15853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a72ea647ed414da1325abff9174bad" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40851 } }
I20260806 19:35:14.832859 15855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91a72ea647ed414da1325abff9174bad. Latest consensus state: current_term: 1 leader_uuid: "91a72ea647ed414da1325abff9174bad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a72ea647ed414da1325abff9174bad" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40851 } } }
I20260806 19:35:14.832995 15854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91a72ea647ed414da1325abff9174bad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91a72ea647ed414da1325abff9174bad" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40851 } } }
I20260806 19:35:14.833360 15855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:14.833448 15854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:14.835508 15863 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:35:14.844130 15863 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:35:14.856091 15863 catalog_manager.cc:1347] Generated new cluster ID: 7a64acad0f9a414181ff529e93a77f75
I20260806 19:35:14.856244 15863 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:35:14.892776 15863 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:35:14.895751 15863 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:35:14.909121 15863 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91a72ea647ed414da1325abff9174bad: Generated new TSK 0
I20260806 19:35:14.912014 15863 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 19:35:14.989673 15879 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 19:35:14.989907 15879 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 19:35:14.992733 15850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:14.953886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59318 (local address 127.10.25.62:40851)
0806 19:35:14.954543 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:14.954554 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:14.955664 (+ 1110us) server_negotiation.cc:407] Connection header received
0806 19:35:14.956186 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:14.956191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:14.956292 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:14.956454 (+ 162us) server_negotiation.cc:226] Negotiated authn=JWT
0806 19:35:14.957644 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:14.958770 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:14.961448 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:14.962380 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:14.962558 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:14.963413 (+ 855us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 19:35:14.991739 (+ 28326us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43537/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 19:35:14.992161 (+ 422us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59318: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43537/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":460}
W20260806 19:35:14.993042 15850 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59318: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43537/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 19:35:14.996984 15878 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:14.953780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:40851 (local address 127.0.0.1:59318)
0806 19:35:14.955428 (+ 1648us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:14.955453 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:14.955707 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:14.956519 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:14.956527 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:14.956546 (+ 19us) client_negotiation.cc:189] Negotiated authn=JWT
0806 19:35:14.957001 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:14.957014 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:14.959190 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:14.959199 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:14.961120 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:14.961143 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:14.962659 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:14.962667 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:14.962894 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:14.963002 (+ 108us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 19:35:14.992476 (+ 29474us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43537/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 19:35:14.993241 (+ 765us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.25.62:40851: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43537/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":1401,"thread_start_us":971,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.10.25.62:40851: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43537/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 19:35:15.002027 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 15778
I20260806 19:35:15.003827 15842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:15.146420 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 15778
2026-08-06T19:35:15Z chronyd exiting
I20260806 19:35:15.202983 10340 test_util.cc:175] -----------------------------------------------
I20260806 19:35:15.203169 10340 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786044861763400-10340-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (1077 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 19:35:15.214211 10340 mini_oidc.cc:152] Starting JWKS server
I20260806 19:35:15.216586 10340 webserver.cc:460] Webserver started at https://127.0.0.1:36729/ using document root <none> and password file <none>
I20260806 19:35:15.216871 10340 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 19:35:15.217818 10340 webserver.cc:460] Webserver started at http://127.0.0.1:44423/ using document root <none> and password file <none>
2026-08-06T19:35:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:35:15Z Disabled control of system clock
I20260806 19:35:15.259836 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:44969
--webserver_interface=127.10.25.62
--webserver_port=0
--builtin_ntp_servers=127.10.25.20:39299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:44969
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36729/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/testchainca.cert with env {}
W20260806 19:35:15.744011 15887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:15.744400 15887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:15.744576 15887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:15.763134 15887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:35:15.763322 15887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:15.763365 15887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:15.763476 15887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:35:15.763556 15887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 19:35:15.763626 15887 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 19:35:15.763672 15887 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36729/jwks.json
I20260806 19:35:15.786652 15887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:44969
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:44969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36729/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:15.790149 15887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:15.792683 15887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:15.823824 15892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:15.823882 15895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:15.823884 15893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:15.825814 15887 server_base.cc:1034] running on GCE node
I20260806 19:35:15.827544 15887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:15.831696 15887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:15.840437 15887 hybrid_clock.cc:648] HybridClock initialized: now 1786044915839871 us; error 411 us; skew 500 ppm
I20260806 19:35:15.841909 15887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:35:15.846817 15887 webserver.cc:460] Webserver started at http://127.10.25.62:36987/ using document root <none> and password file <none>
I20260806 19:35:15.847810 15887 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:15.847990 15887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:15.848421 15887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:15.852484 15887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "70201ad51c7c464586f034f92f71e2ff"
format_stamp: "Formatted at 2026-08-06 19:35:15 on dist-test-slave-ss4n"
server_key: "eb570393f0bd6a7598c6118f9ce929d4"
server_key_iv: "093673ba7ca4ad7e2b03b8552b116869"
server_key_version: "encryptionkey@0"
I20260806 19:35:15.854591 15887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "70201ad51c7c464586f034f92f71e2ff"
format_stamp: "Formatted at 2026-08-06 19:35:15 on dist-test-slave-ss4n"
server_key: "eb570393f0bd6a7598c6118f9ce929d4"
server_key_iv: "093673ba7ca4ad7e2b03b8552b116869"
server_key_version: "encryptionkey@0"
I20260806 19:35:15.866902 15887 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.011s sys 0.001s
I20260806 19:35:15.880898 15901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:15.883819 15887 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.010s sys 0.002s
I20260806 19:35:15.884097 15887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "70201ad51c7c464586f034f92f71e2ff"
format_stamp: "Formatted at 2026-08-06 19:35:15 on dist-test-slave-ss4n"
server_key: "eb570393f0bd6a7598c6118f9ce929d4"
server_key_iv: "093673ba7ca4ad7e2b03b8552b116869"
server_key_version: "encryptionkey@0"
I20260806 19:35:15.884373 15887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:15.925292 15887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:15.926580 15887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:15.927012 15887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:15.985258 15887 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:44969
I20260806 19:35:15.985203 15955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:44969 every 8 connection(s)
I20260806 19:35:15.987255 15887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:35:15.990123 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 15887
I20260806 19:35:15.990384 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:35:15.992494 10340 external_mini_cluster.cc:1428] Setting key c17d29b9da97405fb2ec3ba5b6c303fe
I20260806 19:35:15.999030 15957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:16.023154 15957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff: Bootstrap starting.
I20260806 19:35:16.025808 15960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:15.999083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35158 (local address 127.10.25.62:44969)
0806 19:35:16.001900 (+ 2817us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:16.001918 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:16.001993 (+ 75us) server_negotiation.cc:407] Connection header received
0806 19:35:16.004513 (+ 2520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:16.004543 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:16.005168 (+ 625us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:16.005738 (+ 570us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:16.006458 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:16.009852 (+ 3394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:16.013346 (+ 3494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:16.015576 (+ 2230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:16.015799 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:16.019215 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:16.020019 (+ 804us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:16.020077 (+ 58us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:16.021260 (+ 1183us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:16.021355 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:16.021405 (+ 50us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:16.022279 (+ 874us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:16.022444 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 19:35:16.023293 (+ 849us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1996,"thread_start_us":959,"threads_started":1}
I20260806 19:35:16.027904 15957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:16.029309 15957 log.cc:824] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:16.032797 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:35:16.037940 15957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff: No bootstrap required, opened a new log
I20260806 19:35:16.055110 15957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70201ad51c7c464586f034f92f71e2ff" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44969 } }
I20260806 19:35:16.055938 15957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:16.056175 15957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70201ad51c7c464586f034f92f71e2ff, State: Initialized, Role: FOLLOWER
I20260806 19:35:16.058848 15957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70201ad51c7c464586f034f92f71e2ff" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44969 } }
I20260806 19:35:16.060015 15957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:35:16.060288 15957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:35:16.060531 15957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:16.063514 15957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70201ad51c7c464586f034f92f71e2ff" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44969 } }
I20260806 19:35:16.064338 15957 leader_election.cc:302] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70201ad51c7c464586f034f92f71e2ff; no voters:
I20260806 19:35:16.065256 15957 leader_election.cc:288] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:35:16.065368 15963 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:16.066704 15963 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [term 1 LEADER]: Becoming Leader. State: Replica: 70201ad51c7c464586f034f92f71e2ff, State: Running, Role: LEADER
I20260806 19:35:16.067533 15963 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70201ad51c7c464586f034f92f71e2ff" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44969 } }
I20260806 19:35:16.068594 15957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:35:16.075773 15964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70201ad51c7c464586f034f92f71e2ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70201ad51c7c464586f034f92f71e2ff" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44969 } } }
I20260806 19:35:16.078058 15965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70201ad51c7c464586f034f92f71e2ff. Latest consensus state: current_term: 1 leader_uuid: "70201ad51c7c464586f034f92f71e2ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70201ad51c7c464586f034f92f71e2ff" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44969 } } }
I20260806 19:35:16.081120 15964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:16.081238 15965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:16.082674 15969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:35:16.103811 15969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:35:16.124606 15969 catalog_manager.cc:1347] Generated new cluster ID: 528ce3594d4845ef9d4bfee73663f639
I20260806 19:35:16.124830 15969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:35:16.163797 15969 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:35:16.166625 15969 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:35:16.184636 15969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70201ad51c7c464586f034f92f71e2ff: Generated new TSK 0
I20260806 19:35:16.185968 15969 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 19:35:16.263255 15989 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 19:35:16.263411 15989 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 19:35:16.265468 15960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:16.224830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35174 (local address 127.10.25.62:44969)
0806 19:35:16.225133 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:16.225142 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:16.226051 (+ 909us) server_negotiation.cc:407] Connection header received
0806 19:35:16.226810 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:16.226816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:16.226910 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:16.227094 (+ 184us) server_negotiation.cc:226] Negotiated authn=JWT
0806 19:35:16.227898 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:16.229231 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:16.233999 (+ 4768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:16.238586 (+ 4587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:16.238767 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:16.241646 (+ 2879us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 19:35:16.264597 (+ 22951us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36729/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 19:35:16.265220 (+ 623us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35174: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36729/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":93}
W20260806 19:35:16.266187 15988 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:16.224591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44969 (local address 127.0.0.1:35174)
0806 19:35:16.225716 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:16.225742 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:16.226110 (+ 368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:16.227157 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:16.227164 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:16.227182 (+ 18us) client_negotiation.cc:189] Negotiated authn=JWT
0806 19:35:16.227556 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:16.227570 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:16.229531 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:16.229540 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:16.232072 (+ 2532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:16.232113 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:16.239152 (+ 7039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:16.239165 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:16.240637 (+ 1472us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:16.240747 (+ 110us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 19:35:16.265139 (+ 24392us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36729/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 19:35:16.265524 (+ 385us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.25.62:44969: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36729/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":953,"thread_start_us":470,"threads_started":1}
W20260806 19:35:16.266198 15960 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35174: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36729/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260806 19:35:16.267592 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 15887
I20260806 19:35:16.271397 15951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:16.428999 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 15887
2026-08-06T19:35:16Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (1287 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 19:35:16.497954 10340 mini_oidc.cc:152] Starting JWKS server
I20260806 19:35:16.502012 10340 webserver.cc:460] Webserver started at https://127.0.0.1:44589/ using document root <none> and password file <none>
I20260806 19:35:16.502388 10340 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 19:35:16.503551 10340 webserver.cc:460] Webserver started at http://127.0.0.1:33367/ using document root <none> and password file <none>
2026-08-06T19:35:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:35:16Z Disabled control of system clock
I20260806 19:35:16.559566 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:37491
--webserver_interface=127.10.25.62
--webserver_port=0
--builtin_ntp_servers=127.10.25.20:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:37491
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44589/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 19:35:17.130036 15996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:17.130448 15996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:17.130620 15996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:17.144619 15996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:35:17.144819 15996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:17.144914 15996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:17.145049 15996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:35:17.145139 15996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 19:35:17.145267 15996 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 19:35:17.145352 15996 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44589/jwks.json
I20260806 19:35:17.157804 15996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:37491
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:37491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44589/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:17.159504 15996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:17.161746 15996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:17.178165 16001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:17.178268 16004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:17.178187 16002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:17.179562 15996 server_base.cc:1034] running on GCE node
I20260806 19:35:17.180832 15996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:17.182619 15996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:17.183907 15996 hybrid_clock.cc:648] HybridClock initialized: now 1786044917183881 us; error 61 us; skew 500 ppm
I20260806 19:35:17.184515 15996 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:35:17.187350 15996 webserver.cc:460] Webserver started at http://127.10.25.62:41753/ using document root <none> and password file <none>
I20260806 19:35:17.188259 15996 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:17.188370 15996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:17.188755 15996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:17.192322 15996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "63a3f3f7176648e2b24b088de3fdb9cb"
format_stamp: "Formatted at 2026-08-06 19:35:17 on dist-test-slave-ss4n"
server_key: "d8f48ab3d00869454effb34f848d9370"
server_key_iv: "c1aa8133469c8c9813a41aa21d46c756"
server_key_version: "encryptionkey@0"
I20260806 19:35:17.193339 15996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "63a3f3f7176648e2b24b088de3fdb9cb"
format_stamp: "Formatted at 2026-08-06 19:35:17 on dist-test-slave-ss4n"
server_key: "d8f48ab3d00869454effb34f848d9370"
server_key_iv: "c1aa8133469c8c9813a41aa21d46c756"
server_key_version: "encryptionkey@0"
I20260806 19:35:17.199966 15996 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 19:35:17.206393 16010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:17.210659 15996 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.006s sys 0.000s
I20260806 19:35:17.210999 15996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "63a3f3f7176648e2b24b088de3fdb9cb"
format_stamp: "Formatted at 2026-08-06 19:35:17 on dist-test-slave-ss4n"
server_key: "d8f48ab3d00869454effb34f848d9370"
server_key_iv: "c1aa8133469c8c9813a41aa21d46c756"
server_key_version: "encryptionkey@0"
I20260806 19:35:17.211689 15996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:17.243815 15996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:17.244933 15996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:17.245352 15996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:17.271080 15996 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:37491
I20260806 19:35:17.271098 16062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:37491 every 8 connection(s)
I20260806 19:35:17.272826 15996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:35:17.276887 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 15996
I20260806 19:35:17.277170 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:35:17.277976 10340 external_mini_cluster.cc:1428] Setting key f2dea099fa22436f64d59965aea7b95a
I20260806 19:35:17.279649 16063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:17.301056 16066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:17.280763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50728 (local address 127.10.25.62:37491)
0806 19:35:17.282186 (+ 1423us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:17.282206 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:17.282271 (+ 65us) server_negotiation.cc:407] Connection header received
0806 19:35:17.284285 (+ 2014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:17.284322 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:17.285160 (+ 838us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:17.285815 (+ 655us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:17.286763 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:17.289259 (+ 2496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:17.291344 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:17.292545 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:17.293124 (+ 579us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:17.295617 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:17.295658 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:17.295696 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:17.297057 (+ 1361us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:17.297185 (+ 128us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:17.297229 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:17.297419 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:17.298023 (+ 604us) server_negotiation.cc:299] Negotiation successful
0806 19:35:17.298460 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":659,"threads_started":1}
I20260806 19:35:17.302027 16063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb: Bootstrap starting.
I20260806 19:35:17.306856 16063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:17.308259 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:35:17.308292 16063 log.cc:824] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:17.312824 16063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb: No bootstrap required, opened a new log
I20260806 19:35:17.321133 16063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a3f3f7176648e2b24b088de3fdb9cb" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37491 } }
I20260806 19:35:17.321790 16063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:17.322008 16063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a3f3f7176648e2b24b088de3fdb9cb, State: Initialized, Role: FOLLOWER
I20260806 19:35:17.323077 16063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a3f3f7176648e2b24b088de3fdb9cb" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37491 } }
I20260806 19:35:17.323423 16063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:35:17.323617 16063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:35:17.323858 16063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:17.326547 16063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a3f3f7176648e2b24b088de3fdb9cb" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37491 } }
I20260806 19:35:17.327314 16063 leader_election.cc:302] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63a3f3f7176648e2b24b088de3fdb9cb; no voters:
I20260806 19:35:17.328506 16063 leader_election.cc:288] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:35:17.328672 16068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:17.330367 16068 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [term 1 LEADER]: Becoming Leader. State: Replica: 63a3f3f7176648e2b24b088de3fdb9cb, State: Running, Role: LEADER
I20260806 19:35:17.331614 16068 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a3f3f7176648e2b24b088de3fdb9cb" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37491 } }
I20260806 19:35:17.336136 16063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:35:17.336826 16069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63a3f3f7176648e2b24b088de3fdb9cb. Latest consensus state: current_term: 1 leader_uuid: "63a3f3f7176648e2b24b088de3fdb9cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a3f3f7176648e2b24b088de3fdb9cb" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37491 } } }
I20260806 19:35:17.337903 16070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63a3f3f7176648e2b24b088de3fdb9cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63a3f3f7176648e2b24b088de3fdb9cb" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37491 } } }
I20260806 19:35:17.340191 16070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:17.339300 16069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:17.343880 16072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:35:17.352211 16072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:35:17.369277 16072 catalog_manager.cc:1347] Generated new cluster ID: 5f56dfdf61474854b329032ebd93c172
I20260806 19:35:17.369500 16072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:35:17.393915 16072 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:35:17.396651 16072 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:35:17.410393 16072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63a3f3f7176648e2b24b088de3fdb9cb: Generated new TSK 0
I20260806 19:35:17.411548 16072 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:35:17.446014 16066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:17.429957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50734 (local address 127.10.25.62:37491)
0806 19:35:17.430198 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:17.430206 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:17.430933 (+ 727us) server_negotiation.cc:407] Connection header received
0806 19:35:17.431114 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:17.431120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:17.431234 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:17.431455 (+ 221us) server_negotiation.cc:226] Negotiated authn=JWT
0806 19:35:17.432746 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:17.435329 (+ 2583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:17.437972 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:17.438757 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:17.438980 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:17.441643 (+ 2663us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 19:35:17.445138 (+ 3495us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44589/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 19:35:17.445637 (+ 499us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50734: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44589/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":86}
W20260806 19:35:17.446190 16066 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50734: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44589/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 19:35:17.448041 16094 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:17.429860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:37491 (local address 127.0.0.1:50734)
0806 19:35:17.430646 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:17.430665 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:17.430817 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:17.431854 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:17.431861 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:17.431882 (+ 21us) client_negotiation.cc:189] Negotiated authn=JWT
0806 19:35:17.432430 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:17.432446 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:17.435720 (+ 3274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:17.435728 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:17.437449 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:17.437475 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:17.439992 (+ 2517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:17.440004 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:17.440194 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:17.440286 (+ 92us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 19:35:17.445450 (+ 5164us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44589/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 19:35:17.447529 (+ 2079us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.25.62:37491: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44589/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":632,"mutex_wait_us":1603,"thread_start_us":446,"threads_started":1}
I20260806 19:35:17.449432 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 15996
I20260806 19:35:17.451864 16058 generic_service.cc:196] Checking for leaks (request via RPC)
E20260806 19:35:17.462507 16095 webserver.cc:533] Webserver: sslize failed:
E20260806 19:35:17.462680 16095 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 19:35:17.562975 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 15996
2026-08-06T19:35:17Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (1132 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T19:35:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:35:17Z Disabled control of system clock
I20260806 19:35:17.680832 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:38703
--webserver_interface=127.10.25.62
--webserver_port=0
--builtin_ntp_servers=127.10.25.20:43563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:38703
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/testchainca.cert with env {}
W20260806 19:35:18.131788 16100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:18.132175 16100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:18.132278 16100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:18.140583 16100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:35:18.140730 16100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:18.140774 16100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:18.140825 16100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:35:18.140861 16100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 19:35:18.140918 16100 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 19:35:18.140951 16100 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 19:35:18.154731 16100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:43563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:38703
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:38703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:18.156201 16100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:18.158001 16100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:18.173416 16106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:18.173410 16105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:18.174387 16108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:18.174574 16100 server_base.cc:1034] running on GCE node
I20260806 19:35:18.175755 16100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:18.177673 16100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:18.178972 16100 hybrid_clock.cc:648] HybridClock initialized: now 1786044918178895 us; error 68 us; skew 500 ppm
I20260806 19:35:18.179702 16100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:35:18.183777 16100 webserver.cc:460] Webserver started at http://127.10.25.62:39019/ using document root <none> and password file <none>
I20260806 19:35:18.184674 16100 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:18.184839 16100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:18.185194 16100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:18.188529 16100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "920c003df3274985b7ba7c4e404049a7"
format_stamp: "Formatted at 2026-08-06 19:35:18 on dist-test-slave-ss4n"
server_key: "95d02e82780c65415d797cb3d1162224"
server_key_iv: "71543c3a85940a1321b13d6896fce518"
server_key_version: "encryptionkey@0"
I20260806 19:35:18.189575 16100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "920c003df3274985b7ba7c4e404049a7"
format_stamp: "Formatted at 2026-08-06 19:35:18 on dist-test-slave-ss4n"
server_key: "95d02e82780c65415d797cb3d1162224"
server_key_iv: "71543c3a85940a1321b13d6896fce518"
server_key_version: "encryptionkey@0"
I20260806 19:35:18.195796 16100 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 19:35:18.200381 16114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:18.202268 16100 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:35:18.202512 16100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "920c003df3274985b7ba7c4e404049a7"
format_stamp: "Formatted at 2026-08-06 19:35:18 on dist-test-slave-ss4n"
server_key: "95d02e82780c65415d797cb3d1162224"
server_key_iv: "71543c3a85940a1321b13d6896fce518"
server_key_version: "encryptionkey@0"
I20260806 19:35:18.202796 16100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:18.223325 16100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:18.224399 16100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:18.224749 16100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:18.250499 16100 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:38703
I20260806 19:35:18.250511 16166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:38703 every 8 connection(s)
I20260806 19:35:18.252111 16100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:35:18.256306 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 16100
I20260806 19:35:18.256546 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:35:18.257534 16167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:18.257555 10340 external_mini_cluster.cc:1428] Setting key bffa04a852264f6b77535699fb3c080e
I20260806 19:35:18.274749 16167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7: Bootstrap starting.
I20260806 19:35:18.276914 16170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:18.259842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39146 (local address 127.10.25.62:38703)
0806 19:35:18.262028 (+ 2186us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:18.262046 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:18.262116 (+ 70us) server_negotiation.cc:407] Connection header received
0806 19:35:18.264859 (+ 2743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:18.264895 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:18.265684 (+ 789us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:18.266141 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:18.266982 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:18.268457 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:18.270795 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:18.271700 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:18.271879 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:18.273953 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:18.273963 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:18.274019 (+ 56us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:35:18.274650 (+ 631us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:18.274767 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:35:18.274794 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:18.275079 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:18.275444 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 19:35:18.275852 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":949,"thread_start_us":710,"threads_started":1}
I20260806 19:35:18.281867 16167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:18.282655 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:35:18.283581 16167 log.cc:824] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:18.287269 16167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7: No bootstrap required, opened a new log
I20260806 19:35:18.298416 16167 raft_consensus.cc:348] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920c003df3274985b7ba7c4e404049a7" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38703 } }
I20260806 19:35:18.299901 16167 raft_consensus.cc:374] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:18.300052 16167 raft_consensus.cc:729] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 920c003df3274985b7ba7c4e404049a7, State: Initialized, Role: FOLLOWER
I20260806 19:35:18.301136 16167 consensus_queue.cc:260] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920c003df3274985b7ba7c4e404049a7" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38703 } }
I20260806 19:35:18.301414 16167 raft_consensus.cc:388] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:35:18.301679 16167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:35:18.301893 16167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:18.304682 16167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920c003df3274985b7ba7c4e404049a7" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38703 } }
I20260806 19:35:18.305368 16167 leader_election.cc:302] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 920c003df3274985b7ba7c4e404049a7; no voters:
I20260806 19:35:18.306361 16167 leader_election.cc:288] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:35:18.306819 16172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:18.308665 16172 raft_consensus.cc:686] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [term 1 LEADER]: Becoming Leader. State: Replica: 920c003df3274985b7ba7c4e404049a7, State: Running, Role: LEADER
I20260806 19:35:18.309459 16172 consensus_queue.cc:237] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920c003df3274985b7ba7c4e404049a7" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38703 } }
I20260806 19:35:18.311270 16167 sys_catalog.cc:564] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:35:18.313293 16173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "920c003df3274985b7ba7c4e404049a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920c003df3274985b7ba7c4e404049a7" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38703 } } }
I20260806 19:35:18.313287 16174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 920c003df3274985b7ba7c4e404049a7. Latest consensus state: current_term: 1 leader_uuid: "920c003df3274985b7ba7c4e404049a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "920c003df3274985b7ba7c4e404049a7" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38703 } } }
I20260806 19:35:18.314456 16174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:18.314456 16173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:18.320686 16178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:35:18.329780 16178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:35:18.345693 16178 catalog_manager.cc:1347] Generated new cluster ID: c4f8b65a20fb4660ba99751b5bf89ba3
I20260806 19:35:18.345873 16178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:35:18.376607 16178 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:35:18.378793 16178 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:35:18.395247 16178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 920c003df3274985b7ba7c4e404049a7: Generated new TSK 0
I20260806 19:35:18.396338 16178 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:35:18.861302 16170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:18.421677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39162 (local address 127.10.25.62:38703)
0806 19:35:18.421980 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:18.421990 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:18.422428 (+ 438us) server_negotiation.cc:407] Connection header received
0806 19:35:18.422775 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:18.422780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:18.422897 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:18.423204 (+ 307us) server_negotiation.cc:226] Negotiated authn=JWT
0806 19:35:18.423910 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:18.424775 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:18.427165 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:18.427752 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:18.427881 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:18.429154 (+ 1273us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 19:35:18.860671 (+431517us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 19:35:18.861060 (+ 389us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39162: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":135}
W20260806 19:35:18.861497 16170 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39162: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 19:35:18.861899 16197 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:18.421032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:38703 (local address 127.0.0.1:39162)
0806 19:35:18.422261 (+ 1229us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:18.422283 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:18.422473 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:18.423240 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:18.423246 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:18.423261 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0806 19:35:18.423606 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:18.423627 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:18.424955 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:18.424962 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:18.426918 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:18.426940 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:18.427972 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:18.427979 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:18.428216 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:18.428271 (+ 55us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 19:35:18.861112 (+432841us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 19:35:18.861391 (+ 279us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.25.62:38703: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":1033,"thread_start_us":852,"threads_started":1}
I20260806 19:35:18.863726 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 16100
I20260806 19:35:18.865272 16162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:18.996038 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 16100
2026-08-06T19:35:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1412 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (502 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (507 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:35:20 dist-test-slave-ss4n krb5kdc[16212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:35:20 dist-test-slave-ss4n krb5kdc[16212](info): set up 2 sockets
Aug 06 19:35:20 dist-test-slave-ss4n krb5kdc[16212](info): commencing operation
krb5kdc: starting...
W20260806 19:35:21.337327 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.005s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:35:21 dist-test-slave-ss4n krb5kdc[16212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:35:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:35:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:35:21.638926 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:40379
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:38775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:40379
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:22.044394 16226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:22.044767 16226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:22.044973 16226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:22.053258 16226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:35:22.053428 16226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:22.053510 16226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:22.053571 16226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:35:22.053653 16226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:35:22.064898 16226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:38775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:40379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:40379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:22.066457 16226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:22.068840 16226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:22.085752 16232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:22.085845 16234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:22.085667 16231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:22.086728 16226 server_base.cc:1034] running on GCE node
I20260806 19:35:22.087704 16226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:22.089614 16226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:22.091069 16226 hybrid_clock.cc:648] HybridClock initialized: now 1786044922090953 us; error 65 us; skew 500 ppm
I20260806 19:35:22.091787 16226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:22 dist-test-slave-ss4n krb5kdc[16212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044922, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:22.096947 16226 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:35:22.099637 16226 webserver.cc:460] Webserver started at http://127.10.25.62:40305/ using document root <none> and password file <none>
I20260806 19:35:22.100841 16226 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:22.101046 16226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:22.101538 16226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:22.106079 16226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "5d647a10d90a4dc7a71b953496059918"
format_stamp: "Formatted at 2026-08-06 19:35:22 on dist-test-slave-ss4n"
server_key: "cd83f446c580cb4cb8aba839a2cff9b4"
server_key_iv: "4c3d83884e5186edaca2e7c001c92b39"
server_key_version: "encryptionkey@0"
I20260806 19:35:22.107177 16226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "5d647a10d90a4dc7a71b953496059918"
format_stamp: "Formatted at 2026-08-06 19:35:22 on dist-test-slave-ss4n"
server_key: "cd83f446c580cb4cb8aba839a2cff9b4"
server_key_iv: "4c3d83884e5186edaca2e7c001c92b39"
server_key_version: "encryptionkey@0"
I20260806 19:35:22.114107 16226 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260806 19:35:22.119647 16241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:22.121857 16226 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.001s
I20260806 19:35:22.122131 16226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "5d647a10d90a4dc7a71b953496059918"
format_stamp: "Formatted at 2026-08-06 19:35:22 on dist-test-slave-ss4n"
server_key: "cd83f446c580cb4cb8aba839a2cff9b4"
server_key_iv: "4c3d83884e5186edaca2e7c001c92b39"
server_key_version: "encryptionkey@0"
I20260806 19:35:22.122424 16226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:22.147392 16226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:22.154003 16226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:22.154419 16226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:22.181238 16226 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:40379
I20260806 19:35:22.181262 16293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:40379 every 8 connection(s)
I20260806 19:35:22.183159 16226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:35:22.188167 16294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:22.190505 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 16226
I20260806 19:35:22.190760 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:35:22.191744 10340 external_mini_cluster.cc:1428] Setting key e7a9de6cefaae1669281821388e5d39e
I20260806 19:35:22.207280 16294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918: Bootstrap starting.
Aug 06 19:35:22 dist-test-slave-ss4n krb5kdc[16212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:35:22.210865 16294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:22.212738 16294 log.cc:824] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:22.218025 16294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918: No bootstrap required, opened a new log
I20260806 19:35:22.224265 16297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:22.194493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55090 (local address 127.10.25.62:40379)
0806 19:35:22.195569 (+ 1076us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:22.195590 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:22.195671 (+ 81us) server_negotiation.cc:407] Connection header received
0806 19:35:22.197903 (+ 2232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:22.197944 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:22.198705 (+ 761us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:22.199396 (+ 691us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:22.201092 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:22.203410 (+ 2318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:22.205339 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:22.206666 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:22.206904 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:22.211719 (+ 4815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:22.211733 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:22.211756 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:22.211832 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:22.217377 (+ 5545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:22.218989 (+ 1612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:22.219000 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:22.219010 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:22.219285 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:22.220009 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:22.220015 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:22.220019 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:22.220872 (+ 853us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:22.221229 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:22.222028 (+ 799us) server_negotiation.cc:299] Negotiation successful
0806 19:35:22.222627 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":452,"threads_started":1}
I20260806 19:35:22.231115 16294 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d647a10d90a4dc7a71b953496059918" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40379 } }
I20260806 19:35:22.231710 16294 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:22.231834 16294 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d647a10d90a4dc7a71b953496059918, State: Initialized, Role: FOLLOWER
I20260806 19:35:22.232933 16294 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d647a10d90a4dc7a71b953496059918" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40379 } }
I20260806 19:35:22.233431 16294 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:35:22.233654 16294 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:35:22.233889 16294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:22.238287 16294 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d647a10d90a4dc7a71b953496059918" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40379 } }
I20260806 19:35:22.239346 16294 leader_election.cc:302] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d647a10d90a4dc7a71b953496059918; no voters:
I20260806 19:35:22.240221 16294 leader_election.cc:288] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:35:22.240422 16299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:22.241266 16299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [term 1 LEADER]: Becoming Leader. State: Replica: 5d647a10d90a4dc7a71b953496059918, State: Running, Role: LEADER
I20260806 19:35:22.241963 16299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d647a10d90a4dc7a71b953496059918" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40379 } }
I20260806 19:35:22.242991 16294 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:35:22.245411 16301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d647a10d90a4dc7a71b953496059918. Latest consensus state: current_term: 1 leader_uuid: "5d647a10d90a4dc7a71b953496059918" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d647a10d90a4dc7a71b953496059918" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40379 } } }
I20260806 19:35:22.246007 16301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:22.246929 16300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d647a10d90a4dc7a71b953496059918" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d647a10d90a4dc7a71b953496059918" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40379 } } }
I20260806 19:35:22.247344 16300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:22.248750 16307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:35:22.257650 16307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:35:22.271255 16307 catalog_manager.cc:1347] Generated new cluster ID: 54b92fe7cfe94d308ec4f80b5cfcdf22
I20260806 19:35:22.271406 16307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:35:22.300323 16307 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:35:22.303156 16307 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:35:22.311178 16307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d647a10d90a4dc7a71b953496059918: Generated new TSK 0
I20260806 19:35:22.312502 16307 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:35:22.408963 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:40379
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:38775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:22.841898 16322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:22.842260 16322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:22.842419 16322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:22.851032 16322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:22.851219 16322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:22.851503 16322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:35:22.864301 16322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:38775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:40379
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:22.866070 16322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:22.867739 16322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:22.887337 16328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:22.887616 16327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:22.887782 16330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:22.891729 16322 server_base.cc:1034] running on GCE node
I20260806 19:35:22.892647 16322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:22.893945 16322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:22.895282 16322 hybrid_clock.cc:648] HybridClock initialized: now 1786044922895204 us; error 82 us; skew 500 ppm
I20260806 19:35:22.895715 16322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:22 dist-test-slave-ss4n krb5kdc[16212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044922, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:22.899700 16322 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:35:22.901960 16322 webserver.cc:460] Webserver started at http://127.10.25.1:43809/ using document root <none> and password file <none>
I20260806 19:35:22.902850 16322 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:22.902977 16322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:22.903277 16322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:22.906773 16322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "f3ca70bb06c74aba92e149c21914a28b"
format_stamp: "Formatted at 2026-08-06 19:35:22 on dist-test-slave-ss4n"
server_key: "7ea54a451f0102435328ae76f33d3de5"
server_key_iv: "ab052290c0671489ff59d7ec0bebc1fd"
server_key_version: "encryptionkey@0"
I20260806 19:35:22.907801 16322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "f3ca70bb06c74aba92e149c21914a28b"
format_stamp: "Formatted at 2026-08-06 19:35:22 on dist-test-slave-ss4n"
server_key: "7ea54a451f0102435328ae76f33d3de5"
server_key_iv: "ab052290c0671489ff59d7ec0bebc1fd"
server_key_version: "encryptionkey@0"
I20260806 19:35:22.915345 16322 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 19:35:22.920719 16338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:22.923102 16322 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 19:35:22.923424 16322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "f3ca70bb06c74aba92e149c21914a28b"
format_stamp: "Formatted at 2026-08-06 19:35:22 on dist-test-slave-ss4n"
server_key: "7ea54a451f0102435328ae76f33d3de5"
server_key_iv: "ab052290c0671489ff59d7ec0bebc1fd"
server_key_version: "encryptionkey@0"
I20260806 19:35:22.923720 16322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:22.952888 16322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:22.959761 16322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:22.960210 16322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:22.961865 16322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:22.963743 16322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:22.963891 16322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:22.964071 16322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:22.964166 16322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:23.013254 16322 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:33341
I20260806 19:35:23.013314 16453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:33341 every 8 connection(s)
I20260806 19:35:23.014897 16322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:35:23.016430 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 16322
I20260806 19:35:23.016669 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:35:23.017572 10340 external_mini_cluster.cc:1428] Setting key 548f606f352b28697902845cd91717cf
Aug 06 19:35:23 dist-test-slave-ss4n krb5kdc[16212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044922, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:35:23.043198 16456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:23.018791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:43213 (local address 127.10.25.62:40379)
0806 19:35:23.019344 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:23.019348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:23.020998 (+ 1650us) server_negotiation.cc:407] Connection header received
0806 19:35:23.024825 (+ 3827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:23.024830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:23.024924 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:23.025056 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:23.028175 (+ 3119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:23.029224 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:23.031716 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:23.032164 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:23.032319 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:23.036427 (+ 4108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:23.036432 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:23.036436 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:23.036476 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:23.039398 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:23.040466 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:23.040471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:23.040473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:23.040618 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:23.041128 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:23.041132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:23.041134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:23.041427 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:23.041620 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:23.042470 (+ 850us) server_negotiation.cc:299] Negotiation successful
0806 19:35:23.042759 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":312,"threads_started":1}
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
I20260806 19:35:23.044334 16457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:23.019459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:40379 (local address 127.10.25.1:43213)
0806 19:35:23.020769 (+ 1310us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:23.020844 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:23.024183 (+ 3339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:23.025427 (+ 1244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:23.025454 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:23.026544 (+ 1090us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:23.027825 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:23.027863 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:23.029512 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:23.029521 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:23.031494 (+ 1973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:23.031513 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:23.032464 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:23.032472 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:23.032759 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:23.033596 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:23.033623 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:23.036120 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:23.039624 (+ 3504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:23.039636 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:23.039663 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:23.040290 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:23.040748 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:23.040754 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:23.040758 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:23.040981 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:23.041614 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:23.041624 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:23.042053 (+ 429us) client_negotiation.cc:769] Sending connection context
0806 19:35:23.042358 (+ 305us) client_negotiation.cc:240] Negotiation successful
0806 19:35:23.042999 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"thread_start_us":617,"threads_started":1}
I20260806 19:35:23.046960 16454 heartbeater.cc:347] Connected to a master server at 127.10.25.62:40379
I20260806 19:35:23.047405 16454 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
I20260806 19:35:23.048647 16454 heartbeater.cc:511] Master 127.10.25.62:40379 requested a full tablet report, sending...
I20260806 19:35:23.051417 16258 ts_manager.cc:194] Registered new tserver with Master: f3ca70bb06c74aba92e149c21914a28b (127.10.25.1:33341)
I20260806 19:35:23.053192 16258 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:43213
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:35:23.083091 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:40379
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:38775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:23.489980 16460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:23.490298 16460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:23.490399 16460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:23.498246 16460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:23.498381 16460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:23.498498 16460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:35:23.509686 16460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:38775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:40379
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:23.511190 16460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:23.512805 16460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:23.530129 16466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:23.530218 16467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:23.530187 16469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:23.531066 16460 server_base.cc:1034] running on GCE node
I20260806 19:35:23.531862 16460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:23.533020 16460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:23.534307 16460 hybrid_clock.cc:648] HybridClock initialized: now 1786044923534259 us; error 53 us; skew 500 ppm
I20260806 19:35:23.534706 16460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:23 dist-test-slave-ss4n krb5kdc[16212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044923, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:23.538594 16460 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:35:23.540388 16460 webserver.cc:460] Webserver started at http://127.10.25.2:41429/ using document root <none> and password file <none>
I20260806 19:35:23.541190 16460 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:23.541389 16460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:23.541770 16460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:23.545858 16460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "e17165735e7f4d19ab22a10c7ecb3f69"
format_stamp: "Formatted at 2026-08-06 19:35:23 on dist-test-slave-ss4n"
server_key: "c9d299b36d450d14e611d14b5d78e727"
server_key_iv: "9a96b1fe93ed2c05f18de8ebc76cefa9"
server_key_version: "encryptionkey@0"
I20260806 19:35:23.547183 16460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "e17165735e7f4d19ab22a10c7ecb3f69"
format_stamp: "Formatted at 2026-08-06 19:35:23 on dist-test-slave-ss4n"
server_key: "c9d299b36d450d14e611d14b5d78e727"
server_key_iv: "9a96b1fe93ed2c05f18de8ebc76cefa9"
server_key_version: "encryptionkey@0"
I20260806 19:35:23.554180 16460 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260806 19:35:23.559425 16476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:23.561436 16460 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:35:23.561770 16460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "e17165735e7f4d19ab22a10c7ecb3f69"
format_stamp: "Formatted at 2026-08-06 19:35:23 on dist-test-slave-ss4n"
server_key: "c9d299b36d450d14e611d14b5d78e727"
server_key_iv: "9a96b1fe93ed2c05f18de8ebc76cefa9"
server_key_version: "encryptionkey@0"
I20260806 19:35:23.562057 16460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:23.596268 16460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:23.603809 16460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:23.604357 16460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:23.605708 16460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:23.607448 16460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:23.607591 16460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:23.607788 16460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:23.607900 16460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:23.653401 16460 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:43335
I20260806 19:35:23.653558 16590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:43335 every 8 connection(s)
I20260806 19:35:23.654894 16460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:35:23.659749 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 16460
I20260806 19:35:23.660022 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:35:23.660902 10340 external_mini_cluster.cc:1428] Setting key e3f8b399476f273ecc3bfb617752cd0d
Aug 06 19:35:23 dist-test-slave-ss4n krb5kdc[16212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044923, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:35:23.683821 16593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:23.658555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:58361 (local address 127.10.25.62:40379)
0806 19:35:23.659082 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:23.659089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:23.660703 (+ 1614us) server_negotiation.cc:407] Connection header received
0806 19:35:23.662837 (+ 2134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:23.662843 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:23.662952 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:23.663087 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:23.665021 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:23.666434 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:23.669055 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:23.669755 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:23.669938 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:23.675302 (+ 5364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:23.675308 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:23.675312 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:23.675359 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:23.679005 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:23.680720 (+ 1715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:23.680727 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:23.680729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:23.680930 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:23.681565 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:23.681572 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:23.681580 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:23.682005 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:23.682199 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:23.683131 (+ 932us) server_negotiation.cc:299] Negotiation successful
0806 19:35:23.683417 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":252,"threads_started":1}
I20260806 19:35:23.684937 16594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:23.659124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:40379 (local address 127.10.25.2:58361)
0806 19:35:23.660156 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:23.660219 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:23.662499 (+ 2280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:23.663427 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:23.663442 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:23.664108 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:23.664791 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:23.664813 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:23.666744 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:23.666751 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:23.668775 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:23.668804 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:23.670119 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:23.670130 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:23.670623 (+ 493us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:23.671886 (+ 1263us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:23.671933 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:23.674916 (+ 2983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:23.679347 (+ 4431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:23.679374 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:23.679401 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:23.680446 (+ 1045us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:23.681101 (+ 655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:23.681106 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:23.681111 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:23.681364 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:23.682239 (+ 875us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:23.682249 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:23.682670 (+ 421us) client_negotiation.cc:769] Sending connection context
0806 19:35:23.683044 (+ 374us) client_negotiation.cc:240] Negotiation successful
0806 19:35:23.683687 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":480,"threads_started":1}
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
I20260806 19:35:23.687112 16591 heartbeater.cc:347] Connected to a master server at 127.10.25.62:40379
I20260806 19:35:23.687494 16591 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:23.688490 16591 heartbeater.cc:511] Master 127.10.25.62:40379 requested a full tablet report, sending...
I20260806 19:35:23.690690 16258 ts_manager.cc:194] Registered new tserver with Master: e17165735e7f4d19ab22a10c7ecb3f69 (127.10.25.2:43335)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
I20260806 19:35:23.691854 16258 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:58361
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:35:23.726784 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:40379
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:38775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:35:24.058627 16454 heartbeater.cc:503] Master 127.10.25.62:40379 was elected leader, sending a full tablet report...
W20260806 19:35:24.131707 16597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:24.132067 16597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:24.132176 16597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:24.141819 16597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:24.141986 16597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:24.142124 16597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:35:24.154126 16597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:38775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:40379
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:24.155529 16597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:24.156901 16597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:24.175463 16603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:24.175487 16602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:24.175407 16605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:24.177127 16597 server_base.cc:1034] running on GCE node
I20260806 19:35:24.178057 16597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:24.179301 16597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:24.180624 16597 hybrid_clock.cc:648] HybridClock initialized: now 1786044924180524 us; error 45 us; skew 500 ppm
I20260806 19:35:24.181161 16597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:24 dist-test-slave-ss4n krb5kdc[16212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044924, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:24.186558 16597 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:35:24.192778 16597 webserver.cc:460] Webserver started at http://127.10.25.3:46089/ using document root <none> and password file <none>
I20260806 19:35:24.193991 16597 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:24.194402 16597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:24.194932 16597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:24.199375 16597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "b4dba83578aa462eafd0f913a2dd2af0"
format_stamp: "Formatted at 2026-08-06 19:35:24 on dist-test-slave-ss4n"
server_key: "78490f2bb4feca3a31cf8a846a884ef3"
server_key_iv: "3145a10d59a96c84795cc02c06634dd9"
server_key_version: "encryptionkey@0"
I20260806 19:35:24.200470 16597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "b4dba83578aa462eafd0f913a2dd2af0"
format_stamp: "Formatted at 2026-08-06 19:35:24 on dist-test-slave-ss4n"
server_key: "78490f2bb4feca3a31cf8a846a884ef3"
server_key_iv: "3145a10d59a96c84795cc02c06634dd9"
server_key_version: "encryptionkey@0"
I20260806 19:35:24.207453 16597 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.004s
I20260806 19:35:24.213454 16613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:24.215687 16597 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260806 19:35:24.216161 16597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "b4dba83578aa462eafd0f913a2dd2af0"
format_stamp: "Formatted at 2026-08-06 19:35:24 on dist-test-slave-ss4n"
server_key: "78490f2bb4feca3a31cf8a846a884ef3"
server_key_iv: "3145a10d59a96c84795cc02c06634dd9"
server_key_version: "encryptionkey@0"
I20260806 19:35:24.216655 16597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:24.244153 16597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:24.249378 16597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:24.249815 16597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:24.250972 16597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:24.252715 16597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:24.252846 16597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:24.253165 16597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:24.253466 16597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:24.303567 16597 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:43471
I20260806 19:35:24.303690 16727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:43471 every 8 connection(s)
I20260806 19:35:24.305260 16597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:35:24.310205 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 16597
I20260806 19:35:24.310441 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:35:24.311249 10340 external_mini_cluster.cc:1428] Setting key 526325019ed4e0101be5a0ae40a264d9
Aug 06 19:35:24 dist-test-slave-ss4n krb5kdc[16212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044924, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:35:24.337285 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:24.309019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:59623 (local address 127.10.25.62:40379)
0806 19:35:24.309895 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:24.309904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:24.311694 (+ 1790us) server_negotiation.cc:407] Connection header received
0806 19:35:24.314684 (+ 2990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:24.314690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:24.314783 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:24.314920 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:24.317744 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:24.318738 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:24.320986 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:24.321789 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:24.322195 (+ 406us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:24.327538 (+ 5343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:24.327544 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:24.327548 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:24.327634 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:24.331879 (+ 4245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:24.333288 (+ 1409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:24.333296 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:24.333299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:24.333694 (+ 395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:24.334753 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:24.334758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:24.334760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:24.335149 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:24.335297 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:24.336407 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0806 19:35:24.336713 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"spinlock_wait_cycles":38144,"thread_start_us":468,"threads_started":1}
I20260806 19:35:24.338068 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:24.310021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:40379 (local address 127.10.25.3:59623)
0806 19:35:24.311348 (+ 1327us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:24.311424 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:24.314322 (+ 2898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:24.315632 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:24.315650 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:24.316594 (+ 944us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:24.317337 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:24.317369 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:24.319030 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:24.319035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:24.320601 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:24.320621 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:24.322301 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:24.322314 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:24.322733 (+ 419us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:24.323813 (+ 1080us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:24.323876 (+ 63us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:24.327127 (+ 3251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:24.332226 (+ 5099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:24.332239 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:24.332262 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:24.333002 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:24.333958 (+ 956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:24.333964 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:24.333967 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:24.334554 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:24.335364 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:24.335373 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:24.335817 (+ 444us) client_negotiation.cc:769] Sending connection context
0806 19:35:24.336117 (+ 300us) client_negotiation.cc:240] Negotiation successful
0806 19:35:24.336664 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":905,"thread_start_us":619,"threads_started":1}
I20260806 19:35:24.339996 16729 heartbeater.cc:347] Connected to a master server at 127.10.25.62:40379
I20260806 19:35:24.340400 16729 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:24.341427 16729 heartbeater.cc:511] Master 127.10.25.62:40379 requested a full tablet report, sending...
I20260806 19:35:24.343250 16258 ts_manager.cc:194] Registered new tserver with Master: b4dba83578aa462eafd0f913a2dd2af0 (127.10.25.3:43471)
I20260806 19:35:24.344334 16258 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:59623
I20260806 19:35:24.355944 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/test-admin.keytab.
Aug 06 19:35:24 dist-test-slave-ss4n krb5kdc[16212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:24.398456 10340 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 19:35:24.405539 16737 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:24.402045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38950 (local address 127.10.25.1:33341)
0806 19:35:24.402773 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:24.402782 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:24.402815 (+ 33us) server_negotiation.cc:407] Connection header received
0806 19:35:24.402972 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:24.402980 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:24.403232 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:24.403407 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:24.404832 (+ 1425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38950: BlockingRecv error: recv got EOF from 127.0.0.1:38950 (error 108)
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":377,"threads_started":1}
W20260806 19:35:24.405582 10340 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/krb5cc' of type FILE
Aug 06 19:35:24 dist-test-slave-ss4n krb5kdc[16212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:24.407718 10340 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 19:35:24 dist-test-slave-ss4n krb5kdc[16212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:24.409467 10340 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 19:35:24 dist-test-slave-ss4n krb5kdc[16212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:35:24.434851 16737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:24.412148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38956 (local address 127.10.25.1:33341)
0806 19:35:24.412443 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:24.412452 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:24.413055 (+ 603us) server_negotiation.cc:407] Connection header received
0806 19:35:24.413190 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:24.413194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:24.413271 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:24.413377 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:24.414873 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:24.416141 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:24.418785 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:24.419443 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:24.419595 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:24.425854 (+ 6259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:24.425880 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:24.425903 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:24.425976 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:24.430052 (+ 4076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:24.431368 (+ 1316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:24.431382 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:24.431385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:24.431752 (+ 367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:24.432434 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:24.432439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:24.432441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:24.432943 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:24.433220 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:24.433981 (+ 761us) server_negotiation.cc:299] Negotiation successful
0806 19:35:24.434458 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260806 19:35:24.443024 16737 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:24.440855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38960 (local address 127.10.25.1:33341)
0806 19:35:24.441106 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:24.441113 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:24.441241 (+ 128us) server_negotiation.cc:407] Connection header received
0806 19:35:24.441388 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:24.441393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:24.441467 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:24.441655 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:24.442703 (+ 1048us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38960: BlockingRecv error: recv got EOF from 127.0.0.1:38960 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260806 19:35:24.443058 10340 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/krb5cc' of type FILE
Aug 06 19:35:24 dist-test-slave-ss4n krb5kdc[16212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:24.445237 10340 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 19:35:24 dist-test-slave-ss4n krb5kdc[16212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:24.447789 10340 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 19:35:24 dist-test-slave-ss4n krb5kdc[16212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:35:24.470860 16737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:24.450037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38966 (local address 127.10.25.1:33341)
0806 19:35:24.450389 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:24.450400 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:24.450827 (+ 427us) server_negotiation.cc:407] Connection header received
0806 19:35:24.451034 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:24.451039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:24.451129 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:24.451494 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:24.452873 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:24.453790 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:24.455556 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:24.456069 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:24.456203 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:24.462519 (+ 6316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:24.462527 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:24.462532 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:24.462729 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:24.467321 (+ 4592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:24.468339 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:24.468344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:24.468347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:24.468531 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:24.469053 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:24.469057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:24.469060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:24.469411 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:24.469810 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:24.470325 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 19:35:24.470580 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
W20260806 19:35:24.478094 16737 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:24.475496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38980 (local address 127.10.25.1:33341)
0806 19:35:24.475797 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:24.475806 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:24.476156 (+ 350us) server_negotiation.cc:407] Connection header received
0806 19:35:24.476370 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:24.476376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:24.476472 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:24.476588 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:24.477706 (+ 1118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38980: BlockingRecv error: recv got EOF from 127.0.0.1:38980 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
W20260806 19:35:24.478498 10340 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786044861763400-10340-0/krb5kdc/krb5cc' of type FILE
Aug 06 19:35:24 dist-test-slave-ss4n krb5kdc[16212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:24.480741 10340 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 19:35:24 dist-test-slave-ss4n krb5kdc[16212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:24.482878 10340 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 19:35:24 dist-test-slave-ss4n krb5kdc[16212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:35:24.502955 16737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:24.484868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38996 (local address 127.10.25.1:33341)
0806 19:35:24.485210 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:24.485218 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:24.485417 (+ 199us) server_negotiation.cc:407] Connection header received
0806 19:35:24.485754 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:24.485760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:24.485845 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:24.485972 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:24.487171 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:24.488011 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:24.489456 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:24.489978 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:24.490156 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:24.496109 (+ 5953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:24.496117 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:24.496122 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:24.496177 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:24.499821 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:24.500718 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:24.500726 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:24.500734 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:24.500881 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:24.501298 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:24.501304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:24.501307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:24.501705 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:24.502055 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:24.502364 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 19:35:24.502607 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 19:35:24.505259 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 16322
I20260806 19:35:24.524210 16737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:24.506489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39002 (local address 127.10.25.1:33341)
0806 19:35:24.506746 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:24.506751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:24.507328 (+ 577us) server_negotiation.cc:407] Connection header received
0806 19:35:24.507526 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:24.507530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:24.507621 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:24.507737 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:24.509214 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:24.510114 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:24.512048 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:24.512578 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:24.512724 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:24.515685 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:24.515692 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:24.515706 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:24.515749 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:24.519578 (+ 3829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:24.520838 (+ 1260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:24.520845 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:24.520848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:24.521087 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:24.522001 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:24.522007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:24.522009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:24.522643 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:24.523248 (+ 605us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:24.523623 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 19:35:24.523882 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 19:35:24.525020 16448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:24.619109 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 16322
I20260806 19:35:24.641166 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 16460
Aug 06 19:35:24 dist-test-slave-ss4n krb5kdc[16212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:35:24.662802 16750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:24.642658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44938 (local address 127.10.25.2:43335)
0806 19:35:24.643508 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:24.643516 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:24.643547 (+ 31us) server_negotiation.cc:407] Connection header received
0806 19:35:24.643618 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:24.643627 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:24.643882 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:24.644095 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:24.645812 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:24.647156 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:24.648899 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:24.649725 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:24.649860 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:24.655088 (+ 5228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:24.655104 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:24.655120 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:24.655183 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:24.659153 (+ 3970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:24.660191 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:24.660201 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:24.660203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:24.660347 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:24.660813 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:24.660819 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:24.660829 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:24.661223 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:24.661628 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:24.661987 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 19:35:24.662324 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":352,"threads_started":1}
I20260806 19:35:24.664386 16586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:24.760401 16591 heartbeater.cc:503] Master 127.10.25.62:40379 was elected leader, sending a full tablet report...
I20260806 19:35:24.761677 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 16460
W20260806 19:35:24.783442 16242 connection.cc:535] server connection from 127.10.25.2:58361 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 19:35:24.783620 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 16597
Aug 06 19:35:24 dist-test-slave-ss4n krb5kdc[16212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:35:24.804965 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:24.784874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34442 (local address 127.10.25.3:43471)
0806 19:35:24.785671 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:24.785679 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:24.785714 (+ 35us) server_negotiation.cc:407] Connection header received
0806 19:35:24.785784 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:24.785791 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:24.786024 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:24.786174 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:24.787345 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:24.788431 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:24.789958 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:24.790608 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:24.790785 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:24.796898 (+ 6113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:24.796920 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:24.796938 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:24.797029 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:24.800617 (+ 3588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:24.802366 (+ 1749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:24.802376 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:24.802379 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:24.802611 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:24.803267 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:24.803272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:24.803274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:24.803743 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:24.804008 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:24.804349 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 19:35:24.804635 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":352,"threads_started":1}
I20260806 19:35:24.806475 16723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:24.897336 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 16597
I20260806 19:35:24.918040 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 16226
I20260806 19:35:24.919376 16289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:25.029270 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 16226
2026-08-06T19:35:25Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (5028 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:35:25 dist-test-slave-ss4n krb5kdc[16758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:35:25 dist-test-slave-ss4n krb5kdc[16758](info): set up 2 sockets
Aug 06 19:35:25 dist-test-slave-ss4n krb5kdc[16758](info): commencing operation
krb5kdc: starting...
W20260806 19:35:26.287235 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:35:26 dist-test-slave-ss4n krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:35:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:35:26Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.62.keytab.
Entry for principal oryx/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.62.keytab.
I20260806 19:35:26.578739 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:39721
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.62.keytab
--principal=oryx/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:39721
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:26.962132 16772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:26.962502 16772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:26.962678 16772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:26.970556 16772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:35:26.970700 16772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:26.970774 16772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:26.970860 16772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:35:26.970927 16772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:35:26.981546 16772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:39721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.62.keytab
--principal=oryx/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:39721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:26.983032 16772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:26.984429 16772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:26.997821 16777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:26.997776 16780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:26.997733 16778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:26.999466 16772 server_base.cc:1034] running on GCE node
I20260806 19:35:27.000456 16772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:27.002317 16772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:27.003598 16772 hybrid_clock.cc:648] HybridClock initialized: now 1786044927003525 us; error 64 us; skew 500 ppm
I20260806 19:35:27.004079 16772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:27 dist-test-slave-ss4n krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044927, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:27.008558 16772 init.cc:377] Logged in from keytab as oryx/127.10.25.62@KRBTEST.COM (short username oryx)
I20260806 19:35:27.010504 16772 webserver.cc:460] Webserver started at http://127.10.25.62:36473/ using document root <none> and password file <none>
I20260806 19:35:27.011312 16772 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:27.011492 16772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:27.011858 16772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:27.015404 16772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "ad1e547ad04f471d9301eae45ffcec91"
format_stamp: "Formatted at 2026-08-06 19:35:27 on dist-test-slave-ss4n"
server_key: "7823e4b16f30b74eded62d5becb81f99"
server_key_iv: "81045c83ac84f6dabfee06a14056ab77"
server_key_version: "encryptionkey@0"
I20260806 19:35:27.016306 16772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "ad1e547ad04f471d9301eae45ffcec91"
format_stamp: "Formatted at 2026-08-06 19:35:27 on dist-test-slave-ss4n"
server_key: "7823e4b16f30b74eded62d5becb81f99"
server_key_iv: "81045c83ac84f6dabfee06a14056ab77"
server_key_version: "encryptionkey@0"
I20260806 19:35:27.022830 16772 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 19:35:27.027925 16788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:27.030215 16772 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 19:35:27.030486 16772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "ad1e547ad04f471d9301eae45ffcec91"
format_stamp: "Formatted at 2026-08-06 19:35:27 on dist-test-slave-ss4n"
server_key: "7823e4b16f30b74eded62d5becb81f99"
server_key_iv: "81045c83ac84f6dabfee06a14056ab77"
server_key_version: "encryptionkey@0"
I20260806 19:35:27.030750 16772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:27.048336 16772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:27.054023 16772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:27.054418 16772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:27.081830 16772 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:39721
I20260806 19:35:27.081852 16841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:39721 every 8 connection(s)
I20260806 19:35:27.083549 16772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:35:27.088133 16842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:27.090531 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 16772
I20260806 19:35:27.090780 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:35:27.091509 10340 external_mini_cluster.cc:1428] Setting key 5209ce9b451a9d64f4fc0771c69235b3
I20260806 19:35:27.104328 16842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91: Bootstrap starting.
I20260806 19:35:27.108771 16842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91: Neither blocks nor log segments found. Creating new log.
Aug 06 19:35:27 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM
I20260806 19:35:27.110308 16842 log.cc:824] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:27.115460 16842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91: No bootstrap required, opened a new log
I20260806 19:35:27.124346 16845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:27.094031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57950 (local address 127.10.25.62:39721)
0806 19:35:27.095044 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:27.095076 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:27.095149 (+ 73us) server_negotiation.cc:407] Connection header received
0806 19:35:27.097924 (+ 2775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:27.097963 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:27.098584 (+ 621us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:27.099139 (+ 555us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:27.100717 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:27.103133 (+ 2416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:27.104926 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:27.106172 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:27.106444 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:27.112622 (+ 6178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:27.112639 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:27.112667 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:27.112726 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:27.117765 (+ 5039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:27.119040 (+ 1275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:27.119051 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:27.119063 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:27.119370 (+ 307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:27.120106 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:27.120112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:27.120116 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:27.121014 (+ 898us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:27.121415 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:27.122077 (+ 662us) server_negotiation.cc:299] Negotiation successful
0806 19:35:27.122658 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":446,"threads_started":1}
I20260806 19:35:27.126996 16842 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } }
I20260806 19:35:27.127460 16842 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:27.127627 16842 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad1e547ad04f471d9301eae45ffcec91, State: Initialized, Role: FOLLOWER
I20260806 19:35:27.128367 16842 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } }
I20260806 19:35:27.128717 16842 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:35:27.128887 16842 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:35:27.129133 16842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:27.132200 16842 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } }
I20260806 19:35:27.132916 16842 leader_election.cc:302] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad1e547ad04f471d9301eae45ffcec91; no voters:
I20260806 19:35:27.133805 16842 leader_election.cc:288] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:35:27.133965 16847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:27.135007 16847 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [term 1 LEADER]: Becoming Leader. State: Replica: ad1e547ad04f471d9301eae45ffcec91, State: Running, Role: LEADER
I20260806 19:35:27.135646 16847 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } }
I20260806 19:35:27.136510 16842 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:35:27.138639 16848 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } }
I20260806 19:35:27.139086 16848 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:27.139659 16849 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad1e547ad04f471d9301eae45ffcec91. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } }
I20260806 19:35:27.140137 16849 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:27.141844 16852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:35:27.147734 16852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:35:27.158385 16852 catalog_manager.cc:1347] Generated new cluster ID: f3614b79fba74957ae460f324c0676da
I20260806 19:35:27.158545 16852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:35:27.197217 16852 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:35:27.199273 16852 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:35:27.208997 16852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91: Generated new TSK 0
I20260806 19:35:27.210249 16852 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.1.keytab.
Entry for principal oryx/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.1.keytab.
I20260806 19:35:27.286521 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39721
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.1.keytab
--principal=oryx/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:27.687175 16869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:27.687541 16869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:27.687661 16869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:27.696972 16869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:27.697191 16869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:27.697329 16869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:35:27.709764 16869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.1.keytab
--principal=oryx/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:27.711225 16869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:27.712756 16869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:27.729735 16875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:27.729734 16877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:27.729974 16874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:27.731909 16869 server_base.cc:1034] running on GCE node
I20260806 19:35:27.732779 16869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:27.734472 16869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:27.735836 16869 hybrid_clock.cc:648] HybridClock initialized: now 1786044927735719 us; error 98 us; skew 500 ppm
I20260806 19:35:27.736423 16869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:27 dist-test-slave-ss4n krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044927, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:27.740856 16869 init.cc:377] Logged in from keytab as oryx/127.10.25.1@KRBTEST.COM (short username oryx)
I20260806 19:35:27.743000 16869 webserver.cc:460] Webserver started at http://127.10.25.1:42971/ using document root <none> and password file <none>
I20260806 19:35:27.743953 16869 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:27.744149 16869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:27.744544 16869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:27.748205 16869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c"
format_stamp: "Formatted at 2026-08-06 19:35:27 on dist-test-slave-ss4n"
server_key: "f65dc55902d8c2b4c5b63614b5465d42"
server_key_iv: "11f4cfaa75d3d8695d5cd7cc7768de8e"
server_key_version: "encryptionkey@0"
I20260806 19:35:27.749258 16869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c"
format_stamp: "Formatted at 2026-08-06 19:35:27 on dist-test-slave-ss4n"
server_key: "f65dc55902d8c2b4c5b63614b5465d42"
server_key_iv: "11f4cfaa75d3d8695d5cd7cc7768de8e"
server_key_version: "encryptionkey@0"
I20260806 19:35:27.756103 16869 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 19:35:27.761233 16884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:27.763444 16869 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 19:35:27.763718 16869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c"
format_stamp: "Formatted at 2026-08-06 19:35:27 on dist-test-slave-ss4n"
server_key: "f65dc55902d8c2b4c5b63614b5465d42"
server_key_iv: "11f4cfaa75d3d8695d5cd7cc7768de8e"
server_key_version: "encryptionkey@0"
I20260806 19:35:27.764186 16869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:27.794318 16869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:27.799705 16869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:27.799998 16869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:27.801088 16869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:27.802728 16869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:27.802809 16869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:27.802901 16869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:27.802947 16869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:27.844388 16869 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:35055
I20260806 19:35:27.844708 16999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:35055 every 8 connection(s)
I20260806 19:35:27.846012 16869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:35:27.852463 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 16869
I20260806 19:35:27.852703 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:35:27.853740 10340 external_mini_cluster.cc:1428] Setting key dc77ef7328f2e89eef9c1c3e9f6c7768
Aug 06 19:35:27 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044927, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM
I20260806 19:35:27.872411 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:27.849117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:49689 (local address 127.10.25.62:39721)
0806 19:35:27.849755 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:27.849760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:27.850777 (+ 1017us) server_negotiation.cc:407] Connection header received
0806 19:35:27.853102 (+ 2325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:27.853108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:27.853220 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:27.853373 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:27.855293 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:27.856187 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:27.858282 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:27.858742 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:27.858907 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:27.863614 (+ 4707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:27.863622 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:27.863626 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:27.863668 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:27.867352 (+ 3684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:27.868442 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:27.868448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:27.868450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:27.868901 (+ 451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:27.870077 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:27.870082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:27.870085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:27.870542 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:27.870744 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:27.871677 (+ 933us) server_negotiation.cc:299] Negotiation successful
0806 19:35:27.871973 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":331,"threads_started":1}
I20260806 19:35:27.873625 17003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:27.849720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.1:49689)
0806 19:35:27.850575 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:27.850629 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:27.852723 (+ 2094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:27.853700 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:27.853710 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:27.854378 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:27.855066 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:27.855097 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:27.856393 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:27.856397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:27.858051 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:27.858069 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:27.859051 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:27.859059 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:27.859350 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:27.860194 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:27.860223 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:27.863209 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:27.867608 (+ 4399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:27.867621 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:27.867654 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:27.868247 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:27.869184 (+ 937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:27.869190 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:27.869193 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:27.869863 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:27.870727 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:27.870738 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:27.871226 (+ 488us) client_negotiation.cc:769] Sending connection context
0806 19:35:27.871540 (+ 314us) client_negotiation.cc:240] Negotiation successful
0806 19:35:27.872193 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":362,"threads_started":1}
I20260806 19:35:27.875662 17000 heartbeater.cc:347] Connected to a master server at 127.10.25.62:39721
I20260806 19:35:27.876036 17000 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:27.876878 17000 heartbeater.cc:511] Master 127.10.25.62:39721 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.10.25.2@KRBTEST.COM; defaulting to no policy
I20260806 19:35:27.879592 16806 ts_manager.cc:194] Registered new tserver with Master: 3dbd5f87907f42d9b9799cd6cdb1bf0c (127.10.25.1:35055)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.25.2@KRBTEST.COM" created.
I20260806 19:35:27.881745 16806 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:49689
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.2.keytab.
Entry for principal oryx/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.2.keytab.
I20260806 19:35:27.919248 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39721
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.2.keytab
--principal=oryx/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:28.312348 17006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:28.312700 17006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:28.312798 17006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:28.320515 17006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:28.320662 17006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:28.320780 17006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:35:28.331583 17006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.2.keytab
--principal=oryx/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:28.332950 17006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:28.334375 17006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:28.350744 17012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:28.350852 17011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:28.350824 17014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:28.351961 17006 server_base.cc:1034] running on GCE node
I20260806 19:35:28.352674 17006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:28.353906 17006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:28.355187 17006 hybrid_clock.cc:648] HybridClock initialized: now 1786044928355143 us; error 50 us; skew 500 ppm
I20260806 19:35:28.355654 17006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:28 dist-test-slave-ss4n krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044928, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:28.360327 17006 init.cc:377] Logged in from keytab as oryx/127.10.25.2@KRBTEST.COM (short username oryx)
I20260806 19:35:28.362408 17006 webserver.cc:460] Webserver started at http://127.10.25.2:37081/ using document root <none> and password file <none>
I20260806 19:35:28.363248 17006 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:28.363417 17006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:28.363814 17006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:28.367453 17006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "e787046186fe47b9bd9091b5a4c3e80d"
format_stamp: "Formatted at 2026-08-06 19:35:28 on dist-test-slave-ss4n"
server_key: "29d7a2131a023ecf8e18820a24a54cce"
server_key_iv: "8a5abd6526c03f8b9572b6494dcc45ac"
server_key_version: "encryptionkey@0"
I20260806 19:35:28.368454 17006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "e787046186fe47b9bd9091b5a4c3e80d"
format_stamp: "Formatted at 2026-08-06 19:35:28 on dist-test-slave-ss4n"
server_key: "29d7a2131a023ecf8e18820a24a54cce"
server_key_iv: "8a5abd6526c03f8b9572b6494dcc45ac"
server_key_version: "encryptionkey@0"
I20260806 19:35:28.375247 17006 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 19:35:28.379788 17021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:28.381646 17006 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 19:35:28.381860 17006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "e787046186fe47b9bd9091b5a4c3e80d"
format_stamp: "Formatted at 2026-08-06 19:35:28 on dist-test-slave-ss4n"
server_key: "29d7a2131a023ecf8e18820a24a54cce"
server_key_iv: "8a5abd6526c03f8b9572b6494dcc45ac"
server_key_version: "encryptionkey@0"
I20260806 19:35:28.382145 17006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:28.407460 17006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:28.412940 17006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:28.413311 17006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:28.414554 17006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:28.416339 17006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:28.416453 17006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:28.416600 17006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:28.416693 17006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:28.462098 17006 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:35235
I20260806 19:35:28.462179 17136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:35235 every 8 connection(s)
I20260806 19:35:28.463635 17006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:35:28.470112 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 17006
I20260806 19:35:28.470485 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:35:28.471118 10340 external_mini_cluster.cc:1428] Setting key 03fd8839302814e5a432a8200e8f66e4
Aug 06 19:35:28 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044928, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM
I20260806 19:35:28.490628 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:28.466860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:37387 (local address 127.10.25.62:39721)
0806 19:35:28.467424 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:28.467429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:28.468539 (+ 1110us) server_negotiation.cc:407] Connection header received
0806 19:35:28.470870 (+ 2331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:28.470876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:28.470985 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:28.471124 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:28.473389 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:28.474485 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:28.476424 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:28.477269 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:28.477441 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:28.482155 (+ 4714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:28.482178 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:28.482182 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:28.482225 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:28.486088 (+ 3863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:28.487424 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:28.487435 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:28.487439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:28.487706 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:28.488346 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:28.488351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:28.488353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:28.488722 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:28.488857 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:28.490066 (+ 1209us) server_negotiation.cc:299] Negotiation successful
0806 19:35:28.490303 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":315,"threads_started":1}
I20260806 19:35:28.492379 17140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:28.467538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.2:37387)
0806 19:35:28.468330 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:28.468382 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:28.470452 (+ 2070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:28.471429 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:28.471442 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:28.472090 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:28.472949 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:28.473105 (+ 156us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:28.474718 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:28.474723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:28.476221 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:28.476236 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:28.477739 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:28.477744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:28.478003 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:28.479020 (+ 1017us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:28.479052 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:28.481804 (+ 2752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:28.486335 (+ 4531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:28.486350 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:28.486375 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:28.487210 (+ 835us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:28.487865 (+ 655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:28.487874 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:28.487882 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:28.488201 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:28.488882 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:28.488895 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:28.489549 (+ 654us) client_negotiation.cc:769] Sending connection context
0806 19:35:28.489927 (+ 378us) client_negotiation.cc:240] Negotiation successful
0806 19:35:28.490624 (+ 697us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":317,"threads_started":1}
I20260806 19:35:28.494344 17137 heartbeater.cc:347] Connected to a master server at 127.10.25.62:39721
I20260806 19:35:28.494783 17137 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:28.495682 17137 heartbeater.cc:511] Master 127.10.25.62:39721 requested a full tablet report, sending...
I20260806 19:35:28.497928 16806 ts_manager.cc:194] Registered new tserver with Master: e787046186fe47b9bd9091b5a4c3e80d (127.10.25.2:35235)
I20260806 19:35:28.499086 16806 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:37387
WARNING: no policy specified for oryx/127.10.25.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.25.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.3.keytab.
Entry for principal oryx/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.3.keytab.
I20260806 19:35:28.543184 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39721
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.3.keytab
--principal=oryx/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:35:28.885751 17000 heartbeater.cc:503] Master 127.10.25.62:39721 was elected leader, sending a full tablet report...
W20260806 19:35:28.931197 17143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:28.931559 17143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:28.931758 17143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:28.940243 17143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:28.940384 17143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:28.940583 17143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:35:28.952335 17143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.3.keytab
--principal=oryx/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:28.954161 17143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:28.955606 17143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:28.972185 17148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:28.971885 17149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:28.972597 17152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:28.973948 17143 server_base.cc:1034] running on GCE node
I20260806 19:35:28.975163 17143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:28.976418 17143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:28.977730 17143 hybrid_clock.cc:648] HybridClock initialized: now 1786044928977660 us; error 69 us; skew 500 ppm
I20260806 19:35:28.978327 17143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:28 dist-test-slave-ss4n krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044928, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:28.982527 17143 init.cc:377] Logged in from keytab as oryx/127.10.25.3@KRBTEST.COM (short username oryx)
I20260806 19:35:28.984423 17143 webserver.cc:460] Webserver started at http://127.10.25.3:44201/ using document root <none> and password file <none>
I20260806 19:35:28.985266 17143 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:28.985446 17143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:28.985841 17143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:28.989423 17143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "e90efef159644406821b0eab58afcf43"
format_stamp: "Formatted at 2026-08-06 19:35:28 on dist-test-slave-ss4n"
server_key: "92ecc404224a8e637e31043a25aa670d"
server_key_iv: "ed807a35fac411c132cc6047a7f8816f"
server_key_version: "encryptionkey@0"
I20260806 19:35:28.990553 17143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "e90efef159644406821b0eab58afcf43"
format_stamp: "Formatted at 2026-08-06 19:35:28 on dist-test-slave-ss4n"
server_key: "92ecc404224a8e637e31043a25aa670d"
server_key_iv: "ed807a35fac411c132cc6047a7f8816f"
server_key_version: "encryptionkey@0"
I20260806 19:35:28.997123 17143 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260806 19:35:29.001554 17159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:29.003454 17143 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:35:29.003690 17143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "e90efef159644406821b0eab58afcf43"
format_stamp: "Formatted at 2026-08-06 19:35:28 on dist-test-slave-ss4n"
server_key: "92ecc404224a8e637e31043a25aa670d"
server_key_iv: "ed807a35fac411c132cc6047a7f8816f"
server_key_version: "encryptionkey@0"
I20260806 19:35:29.003952 17143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:29.035863 17143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:29.041998 17143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:29.042470 17143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:29.043784 17143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:29.045454 17143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:29.045619 17143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 19:35:29.045794 17143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:29.045876 17143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:29.090876 17143 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:45683
I20260806 19:35:29.090909 17275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:45683 every 8 connection(s)
I20260806 19:35:29.092439 17143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:35:29.095705 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 17143
I20260806 19:35:29.095942 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:35:29.096563 10340 external_mini_cluster.cc:1428] Setting key b8c6ee2e0860a449541b2e100f804d27
Aug 06 19:35:29 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044928, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM
I20260806 19:35:29.118232 17278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:29.095472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:59935 (local address 127.10.25.62:39721)
0806 19:35:29.096097 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:29.096106 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:29.097437 (+ 1331us) server_negotiation.cc:407] Connection header received
0806 19:35:29.100158 (+ 2721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:29.100168 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:29.100266 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:29.100366 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:29.102262 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.103188 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.105083 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.105697 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.105821 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:29.110300 (+ 4479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:29.110307 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:29.110311 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:29.110360 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:29.114098 (+ 3738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:29.115022 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:29.115027 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:29.115029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:29.115185 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:29.115802 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:29.115806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:29.115808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:29.116110 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:29.116240 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:29.117639 (+ 1399us) server_negotiation.cc:299] Negotiation successful
0806 19:35:29.117890 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":280,"threads_started":1}
I20260806 19:35:29.119257 17279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:29.096081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.3:59935)
0806 19:35:29.097186 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:29.097272 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:29.099800 (+ 2528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:29.100656 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:29.100666 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:29.101269 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:29.102019 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:29.102042 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.103378 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.103383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:29.104872 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:29.104889 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.105980 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.105987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:29.106247 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:29.107029 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:29.107057 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:29.109965 (+ 2908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:29.114285 (+ 4320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:29.114294 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:29.114310 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:29.114876 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:29.115328 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:29.115332 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:29.115335 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:29.115561 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:29.116393 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:29.116404 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:29.117055 (+ 651us) client_negotiation.cc:769] Sending connection context
0806 19:35:29.117349 (+ 294us) client_negotiation.cc:240] Negotiation successful
0806 19:35:29.117951 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":519,"threads_started":1}
I20260806 19:35:29.121685 17276 heartbeater.cc:347] Connected to a master server at 127.10.25.62:39721
I20260806 19:35:29.122102 17276 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:29.123016 17276 heartbeater.cc:511] Master 127.10.25.62:39721 requested a full tablet report, sending...
I20260806 19:35:29.125674 16806 ts_manager.cc:194] Registered new tserver with Master: e90efef159644406821b0eab58afcf43 (127.10.25.3:45683)
I20260806 19:35:29.126778 16806 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:59935
I20260806 19:35:29.127415 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.25.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.25.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.25.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.61.keytab.
Entry for principal oryx/127.10.25.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.61.keytab.
I20260806 19:35:29.195153 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.61:37363
--webserver_interface=127.10.25.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.61.keytab
--principal=oryx/127.10.25.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:39721,127.10.25.61:37363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:35:29.502381 17137 heartbeater.cc:503] Master 127.10.25.62:39721 was elected leader, sending a full tablet report...
W20260806 19:35:29.582011 17282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:29.582434 17282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:29.582612 17282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:29.591656 17282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:35:29.591858 17282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:29.591980 17282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:29.592099 17282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:35:29.592195 17282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:35:29.603081 17282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:39721,127.10.25.61:37363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.61.keytab
--principal=oryx/127.10.25.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.61:37363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.25.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
W20260806 19:35:29.603574 17282 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.10.25.62:39721,127.10.25.61:37363'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 19:35:29.605018 17282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:29.607357 17282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:29.622707 17290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:29.622717 17287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:29.622740 17288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:29.623826 17282 server_base.cc:1034] running on GCE node
I20260806 19:35:29.624888 17282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:29.627002 17282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:29.628304 17282 hybrid_clock.cc:648] HybridClock initialized: now 1786044929628221 us; error 43 us; skew 500 ppm
I20260806 19:35:29.628737 17282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:29 dist-test-slave-ss4n krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044929, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:29.632407 17282 init.cc:377] Logged in from keytab as oryx/127.10.25.61@KRBTEST.COM (short username oryx)
I20260806 19:35:29.634593 17282 webserver.cc:460] Webserver started at http://127.10.25.61:37895/ using document root <none> and password file <none>
I20260806 19:35:29.635557 17282 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:29.635669 17282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:29.636041 17282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:29.639894 17282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/data/instance:
uuid: "cdefd62fa4344bdba4e972b3c82124e5"
format_stamp: "Formatted at 2026-08-06 19:35:29 on dist-test-slave-ss4n"
server_key: "41ef00c699c8f3134bfb43c35d17cc65"
server_key_iv: "6750b8db0e4d38cad67f6e569f0eecdf"
server_key_version: "encryptionkey@0"
I20260806 19:35:29.641021 17282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/wal/instance:
uuid: "cdefd62fa4344bdba4e972b3c82124e5"
format_stamp: "Formatted at 2026-08-06 19:35:29 on dist-test-slave-ss4n"
server_key: "41ef00c699c8f3134bfb43c35d17cc65"
server_key_iv: "6750b8db0e4d38cad67f6e569f0eecdf"
server_key_version: "encryptionkey@0"
I20260806 19:35:29.648550 17282 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 19:35:29.654275 17297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:29.656440 17282 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:35:29.656708 17282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/wal
uuid: "cdefd62fa4344bdba4e972b3c82124e5"
format_stamp: "Formatted at 2026-08-06 19:35:29 on dist-test-slave-ss4n"
server_key: "41ef00c699c8f3134bfb43c35d17cc65"
server_key_iv: "6750b8db0e4d38cad67f6e569f0eecdf"
server_key_version: "encryptionkey@0"
I20260806 19:35:29.656957 17282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:29.677184 17282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:29.682370 17282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:29.682628 17282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:29.707362 17282 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.61:37363
I20260806 19:35:29.707520 17351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.61:37363 every 8 connection(s)
I20260806 19:35:29.709651 17282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/data/info.pb
I20260806 19:35:29.713936 17352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:29.715837 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 17282
I20260806 19:35:29.725557 17352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:35:29 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.25.61@KRBTEST.COM
Aug 06 19:35:29 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044929, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.61@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM
I20260806 19:35:29.747174 17355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:29.718451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41818 (local address 127.10.25.61:37363)
0806 19:35:29.719377 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:29.719390 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:29.719439 (+ 49us) server_negotiation.cc:407] Connection header received
0806 19:35:29.721127 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:29.721153 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:29.721720 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:29.722110 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:29.724563 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.726662 (+ 2099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.728274 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.729385 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.729788 (+ 403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:29.736799 (+ 7011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:29.736813 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:29.736832 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:29.736901 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:29.740987 (+ 4086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:29.741942 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:29.741956 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:29.741964 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:29.742142 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:29.742605 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:29.742609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:29.742611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:29.743455 (+ 844us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:29.743801 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:29.744521 (+ 720us) server_negotiation.cc:299] Negotiation successful
0806 19:35:29.745022 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":434,"threads_started":1}
I20260806 19:35:29.750057 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:29.728869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:57956)
0806 19:35:29.729943 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:29.730015 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:29.730194 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:29.730924 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:29.730930 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:29.731629 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:29.731922 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:29.731932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.734114 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.734143 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:29.736210 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:29.736273 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.737287 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.737296 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:29.737621 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:29.738685 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:29.738711 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:29.741607 (+ 2896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:29.746585 (+ 4978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:29.746599 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:29.746606 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:29.747409 (+ 803us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:29.747926 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:29.747933 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:29.747937 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:29.748122 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:29.748925 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:29.748932 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:29.749114 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 19:35:29.749320 (+ 206us) client_negotiation.cc:240] Negotiation successful
0806 19:35:29.749715 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"thread_start_us":629,"threads_started":1}
I20260806 19:35:29.750149 17356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:29.728872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57956 (local address 127.10.25.62:39721)
0806 19:35:29.729527 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:29.729535 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:29.730132 (+ 597us) server_negotiation.cc:407] Connection header received
0806 19:35:29.730341 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:29.730351 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:29.730673 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:29.730838 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:29.732132 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.733888 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.736488 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.736980 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.737165 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:29.741863 (+ 4698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:29.741870 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:29.741873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:29.741923 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:29.746316 (+ 4393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:29.747617 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:29.747622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:29.747625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:29.747781 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:29.748318 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:29.748322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:29.748324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:29.748731 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:29.748908 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:29.749500 (+ 592us) server_negotiation.cc:299] Negotiation successful
0806 19:35:29.749771 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":306,"threads_started":1}
I20260806 19:35:29.752810 17352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:35:29 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044929, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.61@KRBTEST.COM for oryx/127.10.25.61@KRBTEST.COM
I20260806 19:35:29.768630 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:29.753663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:37363 (local address 127.0.0.1:41820)
0806 19:35:29.754027 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:29.754055 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:29.754333 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:29.754763 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:29.754767 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:29.755214 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:29.755364 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:29.755373 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.756392 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.756399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:29.757294 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:29.757305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.758078 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.758084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:29.758274 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:29.759056 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:29.759082 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:29.762334 (+ 3252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:29.766059 (+ 3725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:29.766066 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:29.766071 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:29.766541 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:29.766996 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:29.767000 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:29.767002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:29.767134 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:29.767816 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:29.767820 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:29.767952 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 19:35:29.768095 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 19:35:29.768358 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":5}
I20260806 19:35:29.769023 17355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:29.753852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41820 (local address 127.10.25.61:37363)
0806 19:35:29.754055 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:29.754062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:29.754170 (+ 108us) server_negotiation.cc:407] Connection header received
0806 19:35:29.754563 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:29.754569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:29.754640 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:29.754745 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:29.755527 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.756234 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.757451 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.757905 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.758062 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:29.762574 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:29.762580 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:29.762583 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:29.762625 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:29.765845 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:29.766701 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:29.766706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:29.766709 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:29.766861 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:29.767273 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:29.767278 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:29.767280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:29.767625 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:29.767759 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:29.768191 (+ 432us) server_negotiation.cc:299] Negotiation successful
0806 19:35:29.768572 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":131,"server-negotiator.queue_time_us":82}
I20260806 19:35:29.775585 17352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: Bootstrap starting.
I20260806 19:35:29.780026 17352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:29.781210 17352 log.cc:824] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:29.785166 17352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: No bootstrap required, opened a new log
I20260806 19:35:29.793726 17352 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } }
I20260806 19:35:29.794212 17352 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:29.794353 17352 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdefd62fa4344bdba4e972b3c82124e5, State: Initialized, Role: FOLLOWER
I20260806 19:35:29.795137 17352 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } }
I20260806 19:35:29.796403 17359 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } } }
I20260806 19:35:29.797119 17359 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:35:29.798022 17352 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 19:35:29.811460 17370 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:35:29.811664 17370 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:35:29.816740 17282 master_runner.cc:428] Detected that this master cdefd62fa4344bdba4e972b3c82124e5 is joining an existing cluster
I20260806 19:35:29.816865 17282 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.25.61:37363
I20260806 19:35:29.842026 17356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:29.821654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57964 (local address 127.10.25.62:39721)
0806 19:35:29.821905 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:29.821915 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:29.822697 (+ 782us) server_negotiation.cc:407] Connection header received
0806 19:35:29.823078 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:29.823086 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:29.823208 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:29.823298 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:29.825473 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.826688 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.828950 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.829778 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.829970 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:29.833011 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:29.833021 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:29.833027 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:29.833076 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:29.838060 (+ 4984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:29.839296 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:29.839301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:29.839303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:29.839460 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:29.839939 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:29.839947 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:29.839951 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:29.840387 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:29.840589 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:29.841324 (+ 735us) server_negotiation.cc:299] Negotiation successful
0806 19:35:29.841642 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 19:35:29.842700 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:29.822708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:37363 (local address 127.0.0.1:41824)
0806 19:35:29.823431 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:29.823451 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:29.823587 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:29.823973 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:29.823977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:29.824590 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:29.824970 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:29.824981 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.826720 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.826727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:29.828859 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:29.828877 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.830339 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.830347 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:29.830494 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:29.831308 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:29.831327 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:29.832418 (+ 1091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:29.838286 (+ 5868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:29.838292 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:29.838297 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:29.839101 (+ 804us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:29.839686 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:29.839693 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:29.839697 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:29.839891 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:29.840595 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:29.840600 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:29.840791 (+ 191us) client_negotiation.cc:769] Sending connection context
0806 19:35:29.840998 (+ 207us) client_negotiation.cc:240] Negotiation successful
0806 19:35:29.841390 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":31,"spinlock_wait_cycles":73856,"thread_start_us":398,"threads_started":1}
I20260806 19:35:29.842806 17376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:29.821650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:57964)
0806 19:35:29.822553 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:29.822579 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:29.822849 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:29.823561 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:29.823566 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:29.824492 (+ 926us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:29.825071 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:29.825089 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.827076 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.827082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:29.828699 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:29.828717 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.830021 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.830036 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:29.830261 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:29.831127 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:29.831155 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:29.832730 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:29.838286 (+ 5556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:29.838292 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:29.838297 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:29.839101 (+ 804us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:29.839622 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:29.839630 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:29.839639 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:29.839796 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:29.840680 (+ 884us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:29.840683 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:29.840849 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 19:35:29.841032 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 19:35:29.842335 (+ 1303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":1036,"thread_start_us":491,"threads_started":1}
I20260806 19:35:29.842775 17355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:29.823332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41824 (local address 127.10.25.61:37363)
0806 19:35:29.823543 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:29.823549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:29.823569 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:35:29.823758 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:29.823763 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:29.823832 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:29.823955 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:29.825141 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.826481 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.829010 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.829919 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.830054 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:29.832665 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:29.832678 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:29.832682 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:29.832712 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:29.838100 (+ 5388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:29.839269 (+ 1169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:29.839276 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:29.839281 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:29.839508 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:29.840045 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:29.840051 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:29.840054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:29.840430 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:29.840608 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:29.841123 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 19:35:29.841632 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":183,"server-negotiator.queue_time_us":60}
Aug 06 19:35:29 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044927, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.62@KRBTEST.COM for oryx/127.10.25.61@KRBTEST.COM
I20260806 19:35:29.881335 17378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:29.851438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:37363 (local address 127.0.0.1:41830)
0806 19:35:29.858578 (+ 7140us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:29.858636 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:29.858910 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:29.859535 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:29.859555 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:29.860721 (+ 1166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:29.861114 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:29.861132 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.862824 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.862833 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:29.865347 (+ 2514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:29.865375 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.866706 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.866717 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:29.867113 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:29.868208 (+ 1095us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:29.868236 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:29.871601 (+ 3365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:29.877165 (+ 5564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:29.877194 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:29.877206 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:29.878039 (+ 833us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:29.878767 (+ 728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:29.878772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:29.878776 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:29.878975 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:29.879934 (+ 959us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:29.879946 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:29.880201 (+ 255us) client_negotiation.cc:769] Sending connection context
0806 19:35:29.880429 (+ 228us) client_negotiation.cc:240] Negotiation successful
0806 19:35:29.880858 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6654,"thread_start_us":6383,"threads_started":1}
I20260806 19:35:29.881898 17355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:29.851520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41830 (local address 127.10.25.61:37363)
0806 19:35:29.851719 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:29.851724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:29.858795 (+ 7071us) server_negotiation.cc:407] Connection header received
0806 19:35:29.859191 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:29.859198 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:29.859318 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:29.859452 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:29.861362 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.862602 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.865658 (+ 3056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:29.866412 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:29.866581 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:29.871942 (+ 5361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:29.871954 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:29.871964 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:29.872043 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:29.876871 (+ 4828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:29.878356 (+ 1485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:29.878367 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:29.878374 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:29.878601 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:29.879129 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:29.879138 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:29.879144 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:29.879749 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:29.879955 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:29.880639 (+ 684us) server_negotiation.cc:299] Negotiation successful
0806 19:35:29.880950 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 19:35:29.884871 16806 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.25.61:37363: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5"
member_type: NON_VOTER
last_known_addr {
host: "127.10.25.61"
port: 37363
}
attrs {
promote: true
}
}
dest_uuid: "ad1e547ad04f471d9301eae45ffcec91"
cas_config_opid_index: -1
I20260806 19:35:29.887380 16806 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: NON_VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.10.25.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.25.60@KRBTEST.COM" created.
I20260806 19:35:29.899062 17382 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER cdefd62fa4344bdba4e972b3c82124e5 (127.10.25.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: NON_VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: true } } }
I20260806 19:35:29.900676 17325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:29.905023 17325 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 FOLLOWER]: Refusing update from remote peer ad1e547ad04f471d9301eae45ffcec91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 19:35:29.909897 17382 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.25.61:37363
I20260806 19:35:29.911998 17385 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: NON_VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 19:35:29.913266 17282 master.cc:537] Master@127.10.25.61:37363 shutting down...
W20260806 19:35:29.922838 16790 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 19:35:29.912303 17383 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: NON_VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: true } } }
I20260806 19:35:29.924700 17383 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: This master's current role is: LEADER
W20260806 19:35:29.936589 16790 consensus_peers.cc:487] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 -> Peer cdefd62fa4344bdba4e972b3c82124e5 (127.10.25.61:37363): Couldn't send request to peer cdefd62fa4344bdba4e972b3c82124e5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 19:35:29.948007 17359 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:35:29.948352 17359 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } }
I20260806 19:35:29.949874 17359 leader_election.cc:288] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad1e547ad04f471d9301eae45ffcec91 (127.10.25.62:39721)
I20260806 19:35:29.951148 16816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdefd62fa4344bdba4e972b3c82124e5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad1e547ad04f471d9301eae45ffcec91" is_pre_election: true
I20260806 19:35:29.952586 17299 leader_election.cc:302] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: cdefd62fa4344bdba4e972b3c82124e5; no voters: ad1e547ad04f471d9301eae45ffcec91
I20260806 19:35:29.953133 17359 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.25.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.60.keytab.
Entry for principal oryx/127.10.25.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.60.keytab.
W20260806 19:35:29.953820 17304 tablet.cc:2369] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:35:29.955722 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.60:33869
--webserver_interface=127.10.25.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.60.keytab
--principal=oryx/127.10.25.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:39721,127.10.25.61:37363,127.10.25.60:33869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:35:29.956420 17282 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:29.956887 17282 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:29.957079 17282 tablet_replica.cc:330] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: stopping tablet replica
I20260806 19:35:29.968521 17282 master.cc:552] Master@127.10.25.61:37363 shutdown complete.
I20260806 19:35:29.968703 17282 master_runner.cc:305] Clearing existing system tablet
I20260806 19:35:29.969264 17282 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:29.969373 17282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:29.972656 17282 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 19:35:29.974619 17390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:29.975111 17282 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:29.975396 17282 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 19:35:29.975484 17282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/wal
uuid: "cdefd62fa4344bdba4e972b3c82124e5"
format_stamp: "Formatted at 2026-08-06 19:35:29 on dist-test-slave-ss4n"
server_key: "41ef00c699c8f3134bfb43c35d17cc65"
server_key_iv: "6750b8db0e4d38cad67f6e569f0eecdf"
server_key_version: "encryptionkey@0"
I20260806 19:35:29.981391 17282 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:29.989140 17282 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 19:35:29.989374 17282 log.cc:1196] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 19:35:29.990355 17282 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: Deleting consensus metadata
I20260806 19:35:29.990705 17282 master_runner.cc:315] Copying system tablet from 127.10.25.62:39721
I20260806 19:35:29.994197 17282 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: tablet copy: Beginning tablet copy session from remote peer at address 127.10.25.62:39721
I20260806 19:35:30.023483 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:29.995379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:57976)
0806 19:35:30.006077 (+ 10698us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:30.006103 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:30.006305 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:30.006951 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:30.006963 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:30.007535 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:30.007905 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.007918 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.009692 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.009699 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.011351 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.011369 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.012513 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.012521 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.012713 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.013611 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:30.013637 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:30.014935 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:30.020009 (+ 5074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:30.020017 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:30.020021 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:30.020612 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:30.021216 (+ 604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:30.021222 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:30.021226 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:30.021406 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:30.022224 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:30.022230 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:30.022427 (+ 197us) client_negotiation.cc:769] Sending connection context
0806 19:35:30.022599 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 19:35:30.022986 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10488,"thread_start_us":10344,"threads_started":1}
I20260806 19:35:30.023487 17356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:29.995374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57976 (local address 127.10.25.62:39721)
0806 19:35:29.995570 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:29.995578 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:30.006275 (+ 10697us) server_negotiation.cc:407] Connection header received
0806 19:35:30.006573 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:30.006591 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:30.006742 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:30.006887 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:30.008143 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.009513 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.011575 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.012305 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.012473 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.015253 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:30.015262 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:30.015267 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:30.015316 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:30.019785 (+ 4469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.020823 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.020830 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.020834 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.021056 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.021576 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.021582 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.021585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.022029 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:30.022261 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:30.022770 (+ 509us) server_negotiation.cc:299] Negotiation successful
0806 19:35:30.023048 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 19:35:30.025393 16826 tablet_copy_service.cc:140] P ad1e547ad04f471d9301eae45ffcec91: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cdefd62fa4344bdba4e972b3c82124e5 ({username='oryx', principal='oryx/127.10.25.61@KRBTEST.COM'} at 127.0.0.1:57976)
I20260806 19:35:30.026067 16826 tablet_copy_service.cc:161] P ad1e547ad04f471d9301eae45ffcec91: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cdefd62fa4344bdba4e972b3c82124e5 at {username='oryx', principal='oryx/127.10.25.61@KRBTEST.COM'} at 127.0.0.1:57976: session id = cdefd62fa4344bdba4e972b3c82124e5-00000000000000000000000000000000
I20260806 19:35:30.032135 16826 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 19:35:30.035964 17282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 19:35:30.044117 17282 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: tablet copy: Starting download of 0 data blocks...
I20260806 19:35:30.044611 17282 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: tablet copy: Starting download of 1 WAL segments...
I20260806 19:35:30.049373 17282 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 19:35:30.055228 16826 tablet_copy_service.cc:342] P ad1e547ad04f471d9301eae45ffcec91: Request end of tablet copy session cdefd62fa4344bdba4e972b3c82124e5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.25.61@KRBTEST.COM'} at 127.0.0.1:57976
I20260806 19:35:30.055545 16826 tablet_copy_service.cc:433] P ad1e547ad04f471d9301eae45ffcec91: ending tablet copy session cdefd62fa4344bdba4e972b3c82124e5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cdefd62fa4344bdba4e972b3c82124e5
I20260806 19:35:30.070180 17282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:30.073583 17397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:30.079473 17398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:30.099190 17400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:30.103083 17282 server_base.cc:1034] running on GCE node
I20260806 19:35:30.103711 17282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:30.113888 17282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:30.115509 17282 hybrid_clock.cc:648] HybridClock initialized: now 1786044930115273 us; error 69 us; skew 500 ppm
I20260806 19:35:30.115926 17282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:30 dist-test-slave-ss4n krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044930, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:30.118743 17282 init.cc:377] Logged in from keytab as oryx/127.10.25.61@KRBTEST.COM (short username oryx)
I20260806 19:35:30.126891 17282 webserver.cc:460] Webserver started at http://127.10.25.61:33807/ using document root <none> and password file <none>
I20260806 19:35:30.127342 17282 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:30.127466 17282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:30.130429 17276 heartbeater.cc:503] Master 127.10.25.62:39721 was elected leader, sending a full tablet report...
I20260806 19:35:30.130529 17282 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.002s
I20260806 19:35:30.133908 17407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:30.134513 17282 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 19:35:30.134848 17282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/wal
uuid: "cdefd62fa4344bdba4e972b3c82124e5"
format_stamp: "Formatted at 2026-08-06 19:35:29 on dist-test-slave-ss4n"
server_key: "41ef00c699c8f3134bfb43c35d17cc65"
server_key_iv: "6750b8db0e4d38cad67f6e569f0eecdf"
server_key_version: "encryptionkey@0"
I20260806 19:35:30.135135 17282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:30.147172 17282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:30.152037 17282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:30.195657 17282 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.61:37363
I20260806 19:35:30.195755 17459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.61:37363 every 8 connection(s)
I20260806 19:35:30.196796 17282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-1/data/info.pb
I20260806 19:35:30.206707 17460 sys_catalog.cc:263] Verifying existing consensus state
I20260806 19:35:30.210934 17460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: Bootstrap starting.
I20260806 19:35:30.229086 17460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:35:30.229895 17460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: Bootstrap complete.
I20260806 19:35:30.231014 17460 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: NON_VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: true } }
I20260806 19:35:30.231298 17460 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cdefd62fa4344bdba4e972b3c82124e5, State: Initialized, Role: LEARNER
I20260806 19:35:30.231907 17460 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: NON_VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: true } }
I20260806 19:35:30.233397 17463 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: NON_VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: true } } }
I20260806 19:35:30.233850 17460 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:35:30.234022 17463 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: This master's current role is: LEARNER
I20260806 19:35:30.247421 17474 catalog_manager.cc:1259] Loaded cluster ID: f3614b79fba74957ae460f324c0676da
I20260806 19:35:30.247700 17474 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: loading cluster ID for follower catalog manager: success
I20260806 19:35:30.251577 17474 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: acquiring CA information for follower catalog manager: success
I20260806 19:35:30.254976 17474 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 19:35:30.442337 17475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.412303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:37363 (local address 127.0.0.1:41832)
0806 19:35:30.432135 (+ 19832us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:30.432164 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:30.432364 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:30.432883 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:30.432888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:30.432917 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:30.433120 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.433132 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.435569 (+ 2437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.435578 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.438813 (+ 3235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.438844 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.441080 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.441090 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.441353 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.441447 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 19:35:30.441610 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 19:35:30.441834 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19672,"thread_start_us":20158,"threads_started":1}
I20260806 19:35:30.442484 17476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.412746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41832 (local address 127.10.25.61:37363)
0806 19:35:30.426644 (+ 13898us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:30.426653 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:30.432304 (+ 5651us) server_negotiation.cc:407] Connection header received
0806 19:35:30.432516 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:30.432522 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:30.432643 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:30.432828 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:30.433288 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.435337 (+ 2049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.439098 (+ 3761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.440802 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.441000 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.441334 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:30.441679 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 19:35:30.441863 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13153,"thread_start_us":12939,"threads_started":1}
I20260806 19:35:30.443903 17434 raft_consensus.cc:1203] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260806 19:35:30.444900 17463 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad1e547ad04f471d9301eae45ffcec91. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: NON_VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: true } } }
I20260806 19:35:30.445230 17463 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: This master's current role is: LEARNER
W20260806 19:35:30.537293 17388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:30.537801 17388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:30.537984 17388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:30.548062 17388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:35:30.548270 17388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:30.548400 17388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:30.548527 17388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:35:30.548629 17388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:35:30.560725 17388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:39721,127.10.25.61:37363,127.10.25.60:33869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.60.keytab
--principal=oryx/127.10.25.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.60:33869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.25.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:30.562523 17388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:30.564085 17388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:30.583253 17482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:30.583293 17483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:30.584242 17485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:30.584705 17388 server_base.cc:1034] running on GCE node
I20260806 19:35:30.586221 17388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:30.589025 17388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:30.590378 17388 hybrid_clock.cc:648] HybridClock initialized: now 1786044930590220 us; error 184 us; skew 500 ppm
I20260806 19:35:30.590996 17388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:30 dist-test-slave-ss4n krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044930, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:30.596321 17388 init.cc:377] Logged in from keytab as oryx/127.10.25.60@KRBTEST.COM (short username oryx)
I20260806 19:35:30.598577 17388 webserver.cc:460] Webserver started at http://127.10.25.60:38571/ using document root <none> and password file <none>
I20260806 19:35:30.599640 17388 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:30.599833 17388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:30.600220 17388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:30.605141 17388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/data/instance:
uuid: "9c4294fa83ab48fda6e92f76562d93b5"
format_stamp: "Formatted at 2026-08-06 19:35:30 on dist-test-slave-ss4n"
server_key: "6bcf777df069d213d218ecb5a77c389c"
server_key_iv: "ecf5a1f49db13db5ed4a844ab7ebe297"
server_key_version: "encryptionkey@0"
I20260806 19:35:30.606523 17388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/wal/instance:
uuid: "9c4294fa83ab48fda6e92f76562d93b5"
format_stamp: "Formatted at 2026-08-06 19:35:30 on dist-test-slave-ss4n"
server_key: "6bcf777df069d213d218ecb5a77c389c"
server_key_iv: "ecf5a1f49db13db5ed4a844ab7ebe297"
server_key_version: "encryptionkey@0"
I20260806 19:35:30.616034 17388 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.001s
I20260806 19:35:30.622444 17492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:30.625367 17388 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.002s
I20260806 19:35:30.625695 17388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/wal
uuid: "9c4294fa83ab48fda6e92f76562d93b5"
format_stamp: "Formatted at 2026-08-06 19:35:30 on dist-test-slave-ss4n"
server_key: "6bcf777df069d213d218ecb5a77c389c"
server_key_iv: "ecf5a1f49db13db5ed4a844ab7ebe297"
server_key_version: "encryptionkey@0"
I20260806 19:35:30.625990 17388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:30.657035 17388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:30.662149 17388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:30.662688 17388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:30.692911 17388 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.60:33869
I20260806 19:35:30.692936 17543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.60:33869 every 8 connection(s)
I20260806 19:35:30.694674 17388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/data/info.pb
I20260806 19:35:30.700479 17544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:30.707062 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 17388
I20260806 19:35:30.715870 17544 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:35:30 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.25.60@KRBTEST.COM
Aug 06 19:35:30 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044930, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.60@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM
I20260806 19:35:30.751786 17547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.711277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33430 (local address 127.10.25.60:33869)
0806 19:35:30.712554 (+ 1277us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:30.712578 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:30.714577 (+ 1999us) server_negotiation.cc:407] Connection header received
0806 19:35:30.717669 (+ 3092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:30.717719 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:30.718414 (+ 695us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:30.718976 (+ 562us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:30.722697 (+ 3721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.725577 (+ 2880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.729324 (+ 3747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.731055 (+ 1731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.731563 (+ 508us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.740968 (+ 9405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:30.740978 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:30.740991 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:30.741045 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:30.745137 (+ 4092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.746434 (+ 1297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.746443 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.746457 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.746678 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.747195 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.747199 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.747201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.748068 (+ 867us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:30.748487 (+ 419us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:30.749097 (+ 610us) server_negotiation.cc:299] Negotiation successful
0806 19:35:30.749983 (+ 886us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":903,"thread_start_us":541,"threads_started":1}
I20260806 19:35:30.758970 17548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.721108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:57990)
0806 19:35:30.736686 (+ 15578us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:30.736758 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:30.737149 (+ 391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:30.738572 (+ 1423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:30.738582 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:30.739618 (+ 1036us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:30.740033 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.740047 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.741315 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.741321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.742794 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.742818 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.744442 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.744455 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.744716 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.745743 (+ 1027us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:30.745770 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:30.749110 (+ 3340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:30.754642 (+ 5532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:30.754654 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:30.754661 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:30.755680 (+ 1019us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:30.756404 (+ 724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:30.756415 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:30.756420 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:30.756734 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:30.757828 (+ 1094us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:30.757837 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:30.758051 (+ 214us) client_negotiation.cc:769] Sending connection context
0806 19:35:30.758271 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 19:35:30.758611 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15220,"thread_start_us":15017,"threads_started":1}
I20260806 19:35:30.759024 17549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.721432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57990 (local address 127.10.25.62:39721)
0806 19:35:30.737973 (+ 16541us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:30.737982 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:30.738012 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:35:30.738156 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:30.738161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:30.738261 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:30.738440 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:30.740283 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.741132 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.742980 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.743970 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.744218 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.749567 (+ 5349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:30.749579 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:30.749585 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:30.749653 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:30.754260 (+ 4607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.755888 (+ 1628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.755897 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.755903 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.756154 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.757009 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.757016 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.757019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.757409 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:30.757696 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:30.758506 (+ 810us) server_negotiation.cc:299] Negotiation successful
0806 19:35:30.758741 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16215,"thread_start_us":16697,"threads_started":1}
I20260806 19:35:30.760671 17544 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:35:30 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044930, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.60@KRBTEST.COM for oryx/127.10.25.61@KRBTEST.COM
I20260806 19:35:30.778156 17548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.761366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:37363 (local address 127.0.0.1:41848)
0806 19:35:30.761604 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:30.761621 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:30.761714 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:30.762188 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:30.762192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:30.762601 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:30.762746 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.762754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.763770 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.763782 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.764780 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.764793 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.765620 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.765626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.765838 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.766589 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:30.766609 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:30.769919 (+ 3310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:30.775164 (+ 5245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:30.775169 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:30.775172 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:30.775763 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:30.776257 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:30.776261 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:30.776264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:30.776410 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:30.777123 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:30.777128 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:30.777281 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 19:35:30.777420 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 19:35:30.777881 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 19:35:30.778178 17476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.761705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41848 (local address 127.10.25.61:37363)
0806 19:35:30.761903 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:30.761907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:30.761923 (+ 16us) server_negotiation.cc:407] Connection header received
0806 19:35:30.761972 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:30.761975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:30.762033 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:30.762147 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:30.762905 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.763613 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.764943 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.765379 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.765566 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.770305 (+ 4739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:30.770311 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:30.770315 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:30.770361 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:30.774906 (+ 4545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.775945 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.775951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.775954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.776128 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.776565 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.776570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.776573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.776968 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:30.777128 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:30.777676 (+ 548us) server_negotiation.cc:299] Negotiation successful
0806 19:35:30.777927 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 19:35:30.779294 17544 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:35:30 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044930, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.60@KRBTEST.COM for oryx/127.10.25.60@KRBTEST.COM
I20260806 19:35:30.796654 17548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.780174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:33869 (local address 127.0.0.1:33442)
0806 19:35:30.780387 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:30.780445 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:30.780569 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:30.780923 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:30.780928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:30.781381 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:30.781606 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.781616 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.782677 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.782685 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.783632 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.783643 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.784438 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.784445 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.784906 (+ 461us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.786031 (+ 1125us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:30.786057 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:30.789840 (+ 3783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:30.793986 (+ 4146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:30.793992 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:30.793995 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:30.794482 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:30.794906 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:30.794910 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:30.794913 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:30.795082 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:30.795817 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:30.795820 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:30.795964 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 19:35:30.796112 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 19:35:30.796350 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 19:35:30.797024 17547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.780358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33442 (local address 127.10.25.60:33869)
0806 19:35:30.780548 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:30.780554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:30.780573 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:35:30.780720 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:30.780725 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:30.780794 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:30.780908 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:30.781751 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.782531 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.783800 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.784241 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.784391 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.790142 (+ 5751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:30.790152 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:30.790157 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:30.790207 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:30.793769 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.794625 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.794630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.794632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.794772 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.795262 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.795266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.795277 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.795645 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:30.795776 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:30.796249 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 19:35:30.796540 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":84}
I20260806 19:35:30.804215 17544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: Bootstrap starting.
I20260806 19:35:30.810138 17544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:30.811271 17544 log.cc:824] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:30.815032 17544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: No bootstrap required, opened a new log
I20260806 19:35:30.823662 17544 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } }
I20260806 19:35:30.824061 17544 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:30.824162 17544 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c4294fa83ab48fda6e92f76562d93b5, State: Initialized, Role: FOLLOWER
I20260806 19:35:30.824915 17544 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } }
I20260806 19:35:30.826558 17551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } } }
I20260806 19:35:30.827165 17551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:35:30.828806 17544 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 19:35:30.840335 17562 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:35:30.840534 17562 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:35:30.846843 17388 master_runner.cc:428] Detected that this master 9c4294fa83ab48fda6e92f76562d93b5 is joining an existing cluster
I20260806 19:35:30.847097 17388 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.25.60:33869
I20260806 19:35:30.861794 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 16869
I20260806 19:35:30.882021 17568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.852752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:33869 (local address 127.0.0.1:33454)
0806 19:35:30.853572 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:30.853596 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:30.853884 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:30.854490 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:30.854494 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:30.855390 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:30.855817 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.855830 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.857724 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.857729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.859967 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.859982 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.861436 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.861443 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.861710 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.862799 (+ 1089us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:30.865155 (+ 2356us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:30.866397 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:30.878212 (+ 11815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:30.878219 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:30.878225 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:30.879065 (+ 840us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:30.879639 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:30.879643 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:30.879646 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:30.879873 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:30.880627 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:30.880632 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:30.880849 (+ 217us) client_negotiation.cc:769] Sending connection context
0806 19:35:30.881001 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 19:35:30.881413 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"mutex_wait_us":30,"spinlock_wait_cycles":67968,"thread_start_us":447,"threads_started":1}
I20260806 19:35:30.882550 17549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.852351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58002 (local address 127.10.25.62:39721)
0806 19:35:30.852577 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:30.852584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:30.854263 (+ 1679us) server_negotiation.cc:407] Connection header received
0806 19:35:30.854521 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:30.854528 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:30.854647 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:30.854734 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:30.856512 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.857771 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.860128 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.860890 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.861022 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.868697 (+ 7675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:30.868707 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:30.868716 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:30.868771 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:30.874843 (+ 6072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.879212 (+ 4369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.879219 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.879223 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.879447 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.880027 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.880033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.880036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.880450 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:30.880640 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:30.881400 (+ 760us) server_negotiation.cc:299] Negotiation successful
0806 19:35:30.881767 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 19:35:30.883286 17570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.854872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:37363 (local address 127.0.0.1:41854)
0806 19:35:30.861639 (+ 6767us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:30.861675 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:30.862085 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:30.862813 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:30.862818 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:30.863871 (+ 1053us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:30.864494 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.864517 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.866905 (+ 2388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.866914 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.870554 (+ 3640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.870576 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.872128 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.872136 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.872350 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.873072 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:30.873100 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:30.874547 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:30.879834 (+ 5287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:30.879842 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:30.879846 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:30.880422 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:30.880996 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:30.881001 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:30.881007 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:30.881189 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:30.881992 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:30.882001 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:30.882276 (+ 275us) client_negotiation.cc:769] Sending connection context
0806 19:35:30.882423 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 19:35:30.882784 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6538,"spinlock_wait_cycles":1280,"thread_start_us":6800,"threads_started":1}
I20260806 19:35:30.883653 17547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.852324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33454 (local address 127.10.25.60:33869)
0806 19:35:30.852596 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:30.852607 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:30.853735 (+ 1128us) server_negotiation.cc:407] Connection header received
0806 19:35:30.854073 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:30.854079 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:30.854174 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:30.854262 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:30.856017 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.857441 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.860118 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.860940 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.861033 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.866636 (+ 5603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:30.866643 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:30.866649 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:30.866687 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:30.877950 (+ 11263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.879274 (+ 1324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.879279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.879285 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.879520 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.880014 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.880018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.880021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.880498 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:30.882603 (+ 2105us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:30.882801 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 19:35:30.883082 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 19:35:30.884442 17476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.854484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41854 (local address 127.10.25.61:37363)
0806 19:35:30.854713 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:30.854720 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:30.861789 (+ 7069us) server_negotiation.cc:407] Connection header received
0806 19:35:30.862315 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:30.862321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:30.862420 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:30.862509 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:30.864814 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.866599 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.870829 (+ 4230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.871602 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.871736 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.874844 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:30.874852 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:30.874858 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:30.874909 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:30.879592 (+ 4683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.880615 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.880621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.880625 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.880828 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.881338 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.881343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.881347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.881765 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:30.881923 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:30.883786 (+ 1863us) server_negotiation.cc:299] Negotiation successful
0806 19:35:30.884074 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 19:35:30.884466 17569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.852877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:58002)
0806 19:35:30.854148 (+ 1271us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:30.854171 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:30.854357 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:30.855002 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:30.855007 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:30.855814 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:30.856309 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.856328 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.857980 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.857991 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.859898 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.859922 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.861403 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.861415 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.861652 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.863400 (+ 1748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:30.863427 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:30.864978 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:30.878242 (+ 13264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:30.878247 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:30.878250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:30.878989 (+ 739us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:30.879612 (+ 623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:30.879621 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:30.879626 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:30.879864 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:30.880713 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:30.880718 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:30.880908 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 19:35:30.881045 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 19:35:30.883692 (+ 2647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"mutex_wait_us":2207,"thread_start_us":872,"threads_started":1}
I20260806 19:35:30.897581 17385 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: NON_VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: true } } }
I20260806 19:35:30.898137 17385 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:30.903329 17573 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91: attempting to promote NON_VOTER cdefd62fa4344bdba4e972b3c82124e5 to VOTER
I20260806 19:35:30.904896 17573 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } }
Aug 06 19:35:30 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044927, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.62@KRBTEST.COM for oryx/127.10.25.60@KRBTEST.COM
I20260806 19:35:30.910861 17434 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 LEARNER]: Refusing update from remote peer ad1e547ad04f471d9301eae45ffcec91: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 19:35:30.911875 17572 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
Aug 06 19:35:30 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.25.1@KRBTEST.COM
W20260806 19:35:30.915810 17498 tablet.cc:2369] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:35:30.920369 17573 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, cdefd62fa4344bdba4e972b3c82124e5 (127.10.25.61) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } }
I20260806 19:35:30.921473 17434 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, cdefd62fa4344bdba4e972b3c82124e5 (127.10.25.61) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } }
I20260806 19:35:30.928318 17572 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } }
I20260806 19:35:30.929057 17572 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:30.929766 17463 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } }
I20260806 19:35:30.930136 17463 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:35:30.931839 17572 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } }
I20260806 19:35:30.932399 17572 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:30.935535 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.863509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44130 (local address 127.10.25.1:35055)
0806 19:35:30.885075 (+ 21566us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:30.885085 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:30.885119 (+ 34us) server_negotiation.cc:407] Connection header received
0806 19:35:30.885199 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:30.885209 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:30.885587 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:30.887852 (+ 2265us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:30.888475 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.890680 (+ 2205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.894021 (+ 3341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.895575 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.906557 (+ 10982us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.921330 (+ 14773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:30.921373 (+ 43us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:30.921395 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:30.921454 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:30.929172 (+ 7718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.930801 (+ 1629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.930815 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.930820 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.931184 (+ 364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.931868 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.931874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.931891 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.932542 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:30.932985 (+ 443us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:30.934001 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0806 19:35:30.934462 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21345,"thread_start_us":13891,"threads_started":1}
I20260806 19:35:30.937422 17547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.896966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33458 (local address 127.10.25.60:33869)
0806 19:35:30.897168 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:30.897177 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:30.897198 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:35:30.897676 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:30.897686 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:30.897853 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:30.899600 (+ 1747us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:30.899978 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.901672 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.904328 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.905559 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.906560 (+ 1001us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.910597 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:30.910603 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:30.910607 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:30.910656 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:30.930904 (+ 20248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.932913 (+ 2009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.932923 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.932926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.933148 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:30.933886 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:30.933891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:30.933895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:30.934537 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:30.934716 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:30.936323 (+ 1607us) server_negotiation.cc:299] Negotiation successful
0806 19:35:30.936824 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 19:35:30.937927 16995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:30.938616 17475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:30.895972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:33869 (local address 127.0.0.1:33458)
0806 19:35:30.896501 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:30.896529 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:30.896719 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:30.898052 (+ 1333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:30.898058 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:30.898946 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:30.899353 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.899373 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.901933 (+ 2560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.901942 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.904051 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:30.904075 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:30.905910 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:30.905918 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:30.906132 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:30.907056 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:30.907088 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:30.910284 (+ 3196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:30.931442 (+ 21158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:30.931449 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:30.931454 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:30.932566 (+ 1112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:30.933308 (+ 742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:30.933314 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:30.933317 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:30.933675 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:30.934869 (+ 1194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:30.934887 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:30.935514 (+ 627us) client_negotiation.cc:769] Sending connection context
0806 19:35:30.935966 (+ 452us) client_negotiation.cc:240] Negotiation successful
0806 19:35:30.937926 (+ 1960us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344}
I20260806 19:35:30.941039 16805 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.25.60:33869: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5"
member_type: NON_VOTER
last_known_addr {
host: "127.10.25.60"
port: 33869
}
attrs {
promote: true
}
}
dest_uuid: "ad1e547ad04f471d9301eae45ffcec91"
cas_config_opid_index: 6
I20260806 19:35:30.942474 16805 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: NON_VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: true } }
I20260806 19:35:30.945748 17434 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 FOLLOWER]: Refusing update from remote peer ad1e547ad04f471d9301eae45ffcec91: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 19:35:30.946388 17519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:30.947062 17572 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 19:35:30.952913 17519 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 1 FOLLOWER]: Refusing update from remote peer ad1e547ad04f471d9301eae45ffcec91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 19:35:30.955056 17573 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: NON_VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:30.955782 17385 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 9c4294fa83ab48fda6e92f76562d93b5 (127.10.25.60) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: NON_VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: true } } }
I20260806 19:35:30.957348 17433 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 9c4294fa83ab48fda6e92f76562d93b5 (127.10.25.60) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: NON_VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: true } } }
I20260806 19:35:30.961869 17385 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.25.60:33869
I20260806 19:35:30.963155 17388 master.cc:537] Master@127.10.25.60:33869 shutting down...
I20260806 19:35:30.980515 17388 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:30.981050 17388 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:30.981309 17388 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: stopping tablet replica
I20260806 19:35:30.982535 17463 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: NON_VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: true } } }
I20260806 19:35:30.982904 17463 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:35:30.987082 17385 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: NON_VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: true } } }
I20260806 19:35:30.987746 17385 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:30.987389 17572 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: NON_VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: true } } }
I20260806 19:35:30.988090 17572 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: This master's current role is: LEADER
W20260806 19:35:30.989018 16792 consensus_peers.cc:487] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 -> Peer 9c4294fa83ab48fda6e92f76562d93b5 (127.10.25.60:33869): Couldn't send request to peer 9c4294fa83ab48fda6e92f76562d93b5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 19:35:31.003053 17388 master.cc:552] Master@127.10.25.60:33869 shutdown complete.
I20260806 19:35:31.003221 17388 master_runner.cc:305] Clearing existing system tablet
I20260806 19:35:31.003883 17388 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:31.004046 17388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:31.007686 17388 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.005s sys 0.000s
I20260806 19:35:31.010134 17582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:31.010694 17388 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:31.010987 17388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 19:35:31.011097 17388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/wal
uuid: "9c4294fa83ab48fda6e92f76562d93b5"
format_stamp: "Formatted at 2026-08-06 19:35:30 on dist-test-slave-ss4n"
server_key: "6bcf777df069d213d218ecb5a77c389c"
server_key_iv: "ecf5a1f49db13db5ed4a844ab7ebe297"
server_key_version: "encryptionkey@0"
I20260806 19:35:31.017022 17388 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:31.026086 17388 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 19:35:31.026383 17388 log.cc:1196] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 19:35:31.027346 17388 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: Deleting consensus metadata
I20260806 19:35:31.027675 17388 master_runner.cc:315] Copying system tablet from 127.10.25.62:39721
I20260806 19:35:31.034094 17388 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: tablet copy: Beginning tablet copy session from remote peer at address 127.10.25.62:39721
I20260806 19:35:31.056533 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:31.035145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:58014)
0806 19:35:31.036521 (+ 1376us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:31.036565 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:31.036790 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:31.037408 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:31.037413 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:31.038236 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:31.039016 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:31.039036 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.040811 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.040817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:31.042920 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:31.042942 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.044257 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.044264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:31.044555 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:31.045562 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:31.045588 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:31.046915 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:31.052806 (+ 5891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:31.052813 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:31.052819 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:31.053436 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:31.054060 (+ 624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:31.054066 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:31.054071 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:31.054280 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:31.055239 (+ 959us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:31.055245 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:31.055475 (+ 230us) client_negotiation.cc:769] Sending connection context
0806 19:35:31.055630 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 19:35:31.056039 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1166,"thread_start_us":971,"threads_started":1}
I20260806 19:35:31.057572 17549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:31.035148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58014 (local address 127.10.25.62:39721)
0806 19:35:31.035636 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:31.035644 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:31.036661 (+ 1017us) server_negotiation.cc:407] Connection header received
0806 19:35:31.037121 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:31.037130 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:31.037226 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:31.037355 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:31.039235 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.040615 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.043135 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.044046 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.044201 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:31.047229 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:31.047237 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:31.047242 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:31.047286 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:31.052553 (+ 5267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:31.053626 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:31.053633 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:31.053637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:31.053905 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:31.054441 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:31.054446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:31.054450 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:31.055087 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:31.055260 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:31.055885 (+ 625us) server_negotiation.cc:299] Negotiation successful
0806 19:35:31.056178 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275}
I20260806 19:35:31.058727 16826 tablet_copy_service.cc:140] P ad1e547ad04f471d9301eae45ffcec91: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9c4294fa83ab48fda6e92f76562d93b5 ({username='oryx', principal='oryx/127.10.25.60@KRBTEST.COM'} at 127.0.0.1:58014)
I20260806 19:35:31.059085 16826 tablet_copy_service.cc:161] P ad1e547ad04f471d9301eae45ffcec91: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9c4294fa83ab48fda6e92f76562d93b5 at {username='oryx', principal='oryx/127.10.25.60@KRBTEST.COM'} at 127.0.0.1:58014: session id = 9c4294fa83ab48fda6e92f76562d93b5-00000000000000000000000000000000
I20260806 19:35:31.066084 16826 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 19:35:31.069085 17388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 19:35:31.080895 17388 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: tablet copy: Starting download of 0 data blocks...
I20260806 19:35:31.081439 17388 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: tablet copy: Starting download of 1 WAL segments...
I20260806 19:35:31.088470 17388 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 19:35:31.095934 16826 tablet_copy_service.cc:342] P ad1e547ad04f471d9301eae45ffcec91: Request end of tablet copy session 9c4294fa83ab48fda6e92f76562d93b5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.25.60@KRBTEST.COM'} at 127.0.0.1:58014
I20260806 19:35:31.096223 16826 tablet_copy_service.cc:433] P ad1e547ad04f471d9301eae45ffcec91: ending tablet copy session 9c4294fa83ab48fda6e92f76562d93b5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9c4294fa83ab48fda6e92f76562d93b5
I20260806 19:35:31.114163 17388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:31.118592 17589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:31.123243 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 16869
W20260806 19:35:31.131521 17590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:31.150921 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:35055
--local_ip_for_outbound_sockets=127.10.25.1
--tserver_master_addrs=127.10.25.62:39721
--webserver_port=42971
--webserver_interface=127.10.25.1
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.1.keytab
--principal=oryx/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.25.62:39721,127.10.25.61:37363
--tserver_master_addrs=127.10.25.62:39721,127.10.25.61:37363,127.10.25.60:33869 with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:31.151146 17592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:31.153105 17388 server_base.cc:1034] running on GCE node
I20260806 19:35:31.153707 17388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:31.154464 17388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:31.155692 17388 hybrid_clock.cc:648] HybridClock initialized: now 1786044931155648 us; error 40 us; skew 500 ppm
I20260806 19:35:31.155968 17388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:31 dist-test-slave-ss4n krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044931, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:31.158495 17388 init.cc:377] Logged in from keytab as oryx/127.10.25.60@KRBTEST.COM (short username oryx)
I20260806 19:35:31.162050 17388 webserver.cc:460] Webserver started at http://127.10.25.60:41235/ using document root <none> and password file <none>
I20260806 19:35:31.162549 17388 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:31.162668 17388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:31.166405 17388 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 19:35:31.168776 17600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:31.169380 17388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 19:35:31.169560 17388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/wal
uuid: "9c4294fa83ab48fda6e92f76562d93b5"
format_stamp: "Formatted at 2026-08-06 19:35:30 on dist-test-slave-ss4n"
server_key: "6bcf777df069d213d218ecb5a77c389c"
server_key_iv: "ecf5a1f49db13db5ed4a844ab7ebe297"
server_key_version: "encryptionkey@0"
I20260806 19:35:31.169775 17388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:31.195614 17388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:31.200673 17388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:31.273649 17388 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.60:33869
I20260806 19:35:31.273682 17652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.60:33869 every 8 connection(s)
I20260806 19:35:31.274817 17388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/master-2/data/info.pb
I20260806 19:35:31.281770 17653 sys_catalog.cc:263] Verifying existing consensus state
I20260806 19:35:31.286290 17653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: Bootstrap starting.
I20260806 19:35:31.309571 17653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:35:31.310338 17653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: Bootstrap complete.
I20260806 19:35:31.311887 17653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: NON_VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: true } }
I20260806 19:35:31.312240 17653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9c4294fa83ab48fda6e92f76562d93b5, State: Initialized, Role: LEARNER
I20260806 19:35:31.312551 17653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: NON_VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: true } }
I20260806 19:35:31.313565 17656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: NON_VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: true } } }
I20260806 19:35:31.314031 17653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:35:31.314090 17656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [sys.catalog]: This master's current role is: LEARNER
I20260806 19:35:31.329962 17667 catalog_manager.cc:1259] Loaded cluster ID: f3614b79fba74957ae460f324c0676da
I20260806 19:35:31.330144 17667 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: loading cluster ID for follower catalog manager: success
I20260806 19:35:31.333868 17667 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: acquiring CA information for follower catalog manager: success
I20260806 19:35:31.336810 17667 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 19:35:31.514197 17668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:31.490233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:33869 (local address 127.0.0.1:33466)
0806 19:35:31.505190 (+ 14957us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:31.505214 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:31.505401 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:31.505937 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:31.505940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:31.505966 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:31.506174 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:31.506184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.508731 (+ 2547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.508738 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:31.510736 (+ 1998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:31.510754 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.513087 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.513094 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:31.513343 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:31.513372 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 19:35:31.513541 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 19:35:31.513694 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14816,"thread_start_us":15290,"threads_started":1}
I20260806 19:35:31.514218 17669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:31.490504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33466 (local address 127.10.25.60:33869)
0806 19:35:31.491402 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:31.491411 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:31.505393 (+ 13982us) server_negotiation.cc:407] Connection header received
0806 19:35:31.505571 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:31.505578 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:31.505707 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:31.505899 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:31.506313 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.508541 (+ 2228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.510974 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.512737 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.512979 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:31.513364 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:31.513565 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 19:35:31.513753 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":527,"threads_started":1}
I20260806 19:35:31.516428 17628 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260806 19:35:31.517151 17656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad1e547ad04f471d9301eae45ffcec91. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: NON_VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: true } } }
I20260806 19:35:31.517467 17656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [sys.catalog]: This master's current role is: LEARNER
W20260806 19:35:31.616922 17597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:31.617375 17597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:31.617633 17597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:31.625834 17597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:31.625982 17597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:31.626104 17597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:35:31.637622 17597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.1.keytab
--principal=oryx/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:35055
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=42971
--tserver_master_addrs=127.10.25.62:39721,127.10.25.61:37363,127.10.25.60:33869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:31.639297 17597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:31.640873 17597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:31.657361 17676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:31.658118 17678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:31.657351 17675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:31.658885 17597 server_base.cc:1034] running on GCE node
I20260806 19:35:31.659757 17597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:31.661020 17597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:31.662302 17597 hybrid_clock.cc:648] HybridClock initialized: now 1786044931662241 us; error 49 us; skew 500 ppm
I20260806 19:35:31.662703 17597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:31 dist-test-slave-ss4n krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044931, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:31.667001 17597 init.cc:377] Logged in from keytab as oryx/127.10.25.1@KRBTEST.COM (short username oryx)
I20260806 19:35:31.669652 17597 webserver.cc:460] Webserver started at http://127.10.25.1:42971/ using document root <none> and password file <none>
I20260806 19:35:31.671078 17597 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:31.671309 17597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:31.679675 17597 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 19:35:31.683701 17687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:31.685608 17597 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 19:35:31.685912 17597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c"
format_stamp: "Formatted at 2026-08-06 19:35:27 on dist-test-slave-ss4n"
server_key: "f65dc55902d8c2b4c5b63614b5465d42"
server_key_iv: "11f4cfaa75d3d8695d5cd7cc7768de8e"
server_key_version: "encryptionkey@0"
I20260806 19:35:31.687356 17597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:31.735172 17597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:31.742483 17597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:31.742944 17597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:31.744372 17597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:31.746304 17597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:31.746567 17597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:31.746737 17597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:31.746881 17597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:31.798218 17597 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:35055
I20260806 19:35:31.798492 17802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:35055 every 8 connection(s)
I20260806 19:35:31.799919 17597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:35:31.811887 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 17597
I20260806 19:35:31.812443 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 17006
Aug 06 19:35:31 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044931, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.61@KRBTEST.COM
Aug 06 19:35:31 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044931, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.60@KRBTEST.COM
Aug 06 19:35:31 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044931, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM
Aug 06 19:35:31 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.25.2@KRBTEST.COM
I20260806 19:35:31.850272 17812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:31.817966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:41729 (local address 127.10.25.62:39721)
0806 19:35:31.825404 (+ 7438us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:31.825413 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:31.828448 (+ 3035us) server_negotiation.cc:407] Connection header received
0806 19:35:31.828851 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:31.828858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:31.829159 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:31.829432 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:31.831839 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.833284 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.835804 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.836543 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.836657 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:31.841723 (+ 5066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:31.841734 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:31.841739 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:31.841791 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:31.846530 (+ 4739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:31.847520 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:31.847527 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:31.847532 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:31.847709 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:31.848227 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:31.848232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:31.848236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:31.848632 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:31.848784 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:31.849621 (+ 837us) server_negotiation.cc:299] Negotiation successful
0806 19:35:31.849884 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7241,"thread_start_us":7056,"threads_started":1}
I20260806 19:35:31.850291 17669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:31.804227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:39059 (local address 127.10.25.60:33869)
0806 19:35:31.804503 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:31.804513 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:31.817631 (+ 13118us) server_negotiation.cc:407] Connection header received
0806 19:35:31.821765 (+ 4134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:31.821777 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:31.821921 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:31.822019 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:31.826257 (+ 4238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.827873 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.831913 (+ 4040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.832675 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.832791 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:31.842189 (+ 9398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:31.842198 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:31.842204 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:31.842255 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:31.845985 (+ 3730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:31.847010 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:31.847015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:31.847018 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:31.847162 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:31.847787 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:31.847792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:31.847796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:31.848174 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:31.848325 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:31.849637 (+ 1312us) server_negotiation.cc:299] Negotiation successful
0806 19:35:31.849906 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 19:35:31.850993 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:31.803511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:39405 (local address 127.10.25.61:37363)
0806 19:35:31.807202 (+ 3691us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:31.807207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:31.825461 (+ 18254us) server_negotiation.cc:407] Connection header received
0806 19:35:31.825799 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:31.825808 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:31.825952 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:31.826146 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:31.829467 (+ 3321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.830825 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.833293 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.834039 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.834187 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:31.841200 (+ 7013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:31.841210 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:31.841215 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:31.841268 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:31.845393 (+ 4125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:31.846972 (+ 1579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:31.846981 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:31.846984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:31.847164 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:31.847787 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:31.847792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:31.847796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:31.848183 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:31.848317 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:31.850324 (+ 2007us) server_negotiation.cc:299] Negotiation successful
0806 19:35:31.850593 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3552,"thread_start_us":3414,"threads_started":1}
I20260806 19:35:31.853155 17813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:31.824643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.1:41729)
0806 19:35:31.828058 (+ 3415us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:31.828090 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:31.828639 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:31.829978 (+ 1339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:31.829985 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:31.830800 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:31.831558 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:31.831578 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.833576 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.833582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:31.835570 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:31.835593 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.837049 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.837057 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:31.837304 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:31.838190 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:31.838220 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:31.841348 (+ 3128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:31.846759 (+ 5411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:31.846765 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:31.846769 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:31.847337 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:31.847887 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:31.847892 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:31.847895 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:31.848083 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:31.848879 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:31.848885 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:31.849071 (+ 186us) client_negotiation.cc:769] Sending connection context
0806 19:35:31.850621 (+ 1550us) client_negotiation.cc:240] Negotiation successful
0806 19:35:31.851248 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3214,"mutex_wait_us":35,"spinlock_wait_cycles":2432,"thread_start_us":2825,"threads_started":1}
I20260806 19:35:31.853339 17808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:31.804290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:33869 (local address 127.10.25.1:39059)
0806 19:35:31.817335 (+ 13045us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:31.817395 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:31.821206 (+ 3811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:31.822844 (+ 1638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:31.822865 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:31.824083 (+ 1218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:31.825798 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:31.825842 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.828251 (+ 2409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.828259 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:31.831633 (+ 3374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:31.831651 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.833299 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.833307 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:31.833733 (+ 426us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:31.834979 (+ 1246us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:31.835020 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:31.841849 (+ 6829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:31.846156 (+ 4307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:31.846161 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:31.846165 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:31.846871 (+ 706us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:31.847332 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:31.847336 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:31.847340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:31.847656 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:31.848423 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:31.848432 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:31.849051 (+ 619us) client_negotiation.cc:769] Sending connection context
0806 19:35:31.850524 (+ 1473us) client_negotiation.cc:240] Negotiation successful
0806 19:35:31.851810 (+ 1286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12043,"mutex_wait_us":654,"spinlock_wait_cycles":22912,"thread_start_us":11778,"threads_started":1}
I20260806 19:35:31.853713 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:31.804668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:37363 (local address 127.10.25.1:39405)
0806 19:35:31.817148 (+ 12480us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:31.825249 (+ 8101us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:31.825608 (+ 359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:31.826438 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:31.826444 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:31.828710 (+ 2266us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:31.829132 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:31.829148 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.831095 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.831101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:31.833077 (+ 1976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:31.833091 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.834601 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.834609 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:31.834821 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:31.835581 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:31.835616 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:31.840818 (+ 5202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:31.845690 (+ 4872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:31.845704 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:31.845728 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:31.846803 (+ 1075us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:31.847332 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:31.847336 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:31.847340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:31.847644 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:31.848423 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:31.848432 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:31.849987 (+ 1555us) client_negotiation.cc:769] Sending connection context
0806 19:35:31.851506 (+ 1519us) client_negotiation.cc:240] Negotiation successful
0806 19:35:31.851950 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12237,"mutex_wait_us":68,"spinlock_wait_cycles":38144,"thread_start_us":12516,"threads_started":1}
I20260806 19:35:31.856022 17803 heartbeater.cc:347] Connected to a master server at 127.10.25.60:33869
I20260806 19:35:31.856089 17804 heartbeater.cc:347] Connected to a master server at 127.10.25.62:39721
I20260806 19:35:31.856307 17805 heartbeater.cc:347] Connected to a master server at 127.10.25.61:37363
I20260806 19:35:31.856485 17804 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:31.856515 17805 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:31.856652 17803 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:31.857914 17804 heartbeater.cc:511] Master 127.10.25.62:39721 requested a full tablet report, sending...
I20260806 19:35:31.857951 17803 heartbeater.cc:511] Master 127.10.25.60:33869 requested a full tablet report, sending...
I20260806 19:35:31.858027 17811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:31.813824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38260 (local address 127.10.25.2:35235)
0806 19:35:31.824763 (+ 10939us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:31.824777 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:31.826454 (+ 1677us) server_negotiation.cc:407] Connection header received
0806 19:35:31.827717 (+ 1263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:31.827737 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:31.828299 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:31.828628 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:31.831991 (+ 3363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.833940 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.837292 (+ 3352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:31.838893 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:31.839028 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:31.846381 (+ 7353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:31.846407 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:31.846429 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:31.846491 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:31.851752 (+ 5261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:31.852875 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:31.852884 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:31.852887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:31.853128 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:31.854277 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:31.854287 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:31.854290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:31.854887 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:31.855295 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:31.855871 (+ 576us) server_negotiation.cc:299] Negotiation successful
0806 19:35:31.857532 (+ 1661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10580,"thread_start_us":11610,"threads_started":1}
I20260806 19:35:31.858031 17805 heartbeater.cc:511] Master 127.10.25.61:37363 requested a full tablet report, sending...
I20260806 19:35:31.859644 17132 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:31.860508 16805 ts_manager.cc:194] Re-registered known tserver with Master: 3dbd5f87907f42d9b9799cd6cdb1bf0c (127.10.25.1:35055)
I20260806 19:35:31.860767 17618 ts_manager.cc:194] Registered new tserver with Master: 3dbd5f87907f42d9b9799cd6cdb1bf0c (127.10.25.1:35055)
I20260806 19:35:31.861229 17424 ts_manager.cc:194] Registered new tserver with Master: 3dbd5f87907f42d9b9799cd6cdb1bf0c (127.10.25.1:35055)
I20260806 19:35:31.861788 16805 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:41729
I20260806 19:35:31.908106 17578 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: NON_VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: true } } }
I20260806 19:35:31.908532 17578 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:31.910543 17816 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91: attempting to promote NON_VOTER 9c4294fa83ab48fda6e92f76562d93b5 to VOTER
I20260806 19:35:31.911283 17816 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } }
I20260806 19:35:31.914620 17433 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 FOLLOWER]: Refusing update from remote peer ad1e547ad04f471d9301eae45ffcec91: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 19:35:31.914856 17628 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 1 LEARNER]: Refusing update from remote peer ad1e547ad04f471d9301eae45ffcec91: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 19:35:31.915501 17815 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260806 19:35:31.916384 17816 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 19:35:31.922864 17816 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 9c4294fa83ab48fda6e92f76562d93b5 (127.10.25.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } } }
I20260806 19:35:31.925235 17628 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 9c4294fa83ab48fda6e92f76562d93b5 (127.10.25.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } } }
I20260806 19:35:31.927933 17815 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } } }
I20260806 19:35:31.928336 17815 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:31.928828 17815 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } } }
I20260806 19:35:31.929091 17815 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:31.930251 17656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } } }
I20260806 19:35:31.930548 17656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:35:31.931182 17816 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } } }
I20260806 19:35:31.931536 17816 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:31.932389 17433 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 9c4294fa83ab48fda6e92f76562d93b5 (127.10.25.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } } }
I20260806 19:35:31.938149 17820 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad1e547ad04f471d9301eae45ffcec91" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } } }
I20260806 19:35:31.938755 17820 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:35:31.967893 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 17006
I20260806 19:35:31.991262 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:35235
--local_ip_for_outbound_sockets=127.10.25.2
--tserver_master_addrs=127.10.25.62:39721
--webserver_port=37081
--webserver_interface=127.10.25.2
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.2.keytab
--principal=oryx/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.25.62:39721,127.10.25.61:37363
--tserver_master_addrs=127.10.25.62:39721,127.10.25.61:37363,127.10.25.60:33869 with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:32.410534 17821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:32.410898 17821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:32.411077 17821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:32.420208 17821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:32.420359 17821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:32.420578 17821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:35:32.432847 17821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.2.keytab
--principal=oryx/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:35235
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=37081
--tserver_master_addrs=127.10.25.62:39721,127.10.25.61:37363,127.10.25.60:33869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:32.434466 17821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:32.436025 17821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:32.451573 17826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:32.451551 17829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:32.452102 17827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:32.453528 17821 server_base.cc:1034] running on GCE node
I20260806 19:35:32.454303 17821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:32.455597 17821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:32.456884 17821 hybrid_clock.cc:648] HybridClock initialized: now 1786044932456806 us; error 74 us; skew 500 ppm
I20260806 19:35:32.457379 17821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:32 dist-test-slave-ss4n krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044932, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:32.461615 17821 init.cc:377] Logged in from keytab as oryx/127.10.25.2@KRBTEST.COM (short username oryx)
I20260806 19:35:32.463516 17821 webserver.cc:460] Webserver started at http://127.10.25.2:37081/ using document root <none> and password file <none>
I20260806 19:35:32.464339 17821 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:32.464475 17821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:32.471545 17821 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 19:35:32.475759 17837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:32.478302 17821 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:35:32.478627 17821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "e787046186fe47b9bd9091b5a4c3e80d"
format_stamp: "Formatted at 2026-08-06 19:35:28 on dist-test-slave-ss4n"
server_key: "29d7a2131a023ecf8e18820a24a54cce"
server_key_iv: "8a5abd6526c03f8b9572b6494dcc45ac"
server_key_version: "encryptionkey@0"
I20260806 19:35:32.479839 17821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:32.502501 17821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:32.508194 17821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:32.508527 17821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:32.509835 17821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:32.511490 17821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:32.511650 17821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:32.511811 17821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:32.511905 17821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:32.556056 17821 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:35235
I20260806 19:35:32.556268 17953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:35235 every 8 connection(s)
I20260806 19:35:32.557746 17821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:35:32.566502 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 17821
I20260806 19:35:32.566927 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 17143
Aug 06 19:35:32 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044932, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.61@KRBTEST.COM
Aug 06 19:35:32 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044932, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.60@KRBTEST.COM
Aug 06 19:35:32 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044932, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM
Aug 06 19:35:32 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.25.3@KRBTEST.COM
I20260806 19:35:32.608454 17958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:32.561106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:44849 (local address 127.10.25.61:37363)
0806 19:35:32.562762 (+ 1656us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:32.562770 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:32.563340 (+ 570us) server_negotiation.cc:407] Connection header received
0806 19:35:32.567291 (+ 3951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:32.567300 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:32.567512 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:32.567990 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:32.573967 (+ 5977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:32.575680 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:32.584189 (+ 8509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:32.585120 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:32.585291 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:32.597977 (+ 12686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:32.597985 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:32.597990 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:32.598040 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:32.602339 (+ 4299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:32.603562 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:32.603568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:32.603572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:32.603726 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:32.605056 (+ 1330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:32.605065 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:32.605070 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:32.605733 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:32.605927 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:32.607661 (+ 1734us) server_negotiation.cc:299] Negotiation successful
0806 19:35:32.607940 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1528,"thread_start_us":1397,"threads_started":1}
I20260806 19:35:32.608770 17959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:32.561159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:47731 (local address 127.10.25.60:33869)
0806 19:35:32.562374 (+ 1215us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:32.562383 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:32.563287 (+ 904us) server_negotiation.cc:407] Connection header received
0806 19:35:32.570470 (+ 7183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:32.570481 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:32.570603 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:32.570713 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:32.575995 (+ 5282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:32.577688 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:32.586961 (+ 9273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:32.587924 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:32.588071 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:32.599500 (+ 11429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:32.599508 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:32.599514 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:32.599568 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:32.603820 (+ 4252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:32.605463 (+ 1643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:32.605470 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:32.605475 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:32.605719 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:32.606720 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:32.606727 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:32.606734 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:32.607123 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:32.607262 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:32.607864 (+ 602us) server_negotiation.cc:299] Negotiation successful
0806 19:35:32.608226 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1065,"thread_start_us":904,"threads_started":1}
I20260806 19:35:32.610311 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:32.561819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:33869 (local address 127.10.25.2:47731)
0806 19:35:32.563095 (+ 1276us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:32.563145 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:32.566531 (+ 3386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:32.571023 (+ 4492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:32.571031 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:32.571806 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:32.573022 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:32.573059 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:32.578031 (+ 4972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:32.578040 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:32.581271 (+ 3231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:32.581300 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:32.588521 (+ 7221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:32.588531 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:32.588819 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:32.593669 (+ 4850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:32.593689 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:32.596823 (+ 3134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:32.604018 (+ 7195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:32.604024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:32.604028 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:32.605283 (+ 1255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:32.605897 (+ 614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:32.605904 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:32.605909 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:32.606126 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:32.607376 (+ 1250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:32.607384 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:32.607565 (+ 181us) client_negotiation.cc:769] Sending connection context
0806 19:35:32.609075 (+ 1510us) client_negotiation.cc:240] Negotiation successful
0806 19:35:32.609462 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"thread_start_us":624,"threads_started":1}
I20260806 19:35:32.610349 17961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:32.561819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:37363 (local address 127.10.25.2:44849)
0806 19:35:32.563154 (+ 1335us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:32.563173 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:32.566926 (+ 3753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:32.568754 (+ 1828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:32.568775 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:32.570158 (+ 1383us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:32.573704 (+ 3546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:32.573721 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:32.581613 (+ 7892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:32.581623 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:32.583856 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:32.583883 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:32.585781 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:32.585791 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:32.586241 (+ 450us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:32.590398 (+ 4157us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:32.590461 (+ 63us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:32.597695 (+ 7234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:32.602554 (+ 4859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:32.602566 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:32.602589 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:32.603365 (+ 776us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:32.603877 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:32.603881 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:32.603884 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:32.604790 (+ 906us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:32.606099 (+ 1309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:32.606111 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:32.607014 (+ 903us) client_negotiation.cc:769] Sending connection context
0806 19:35:32.607482 (+ 468us) client_negotiation.cc:240] Negotiation successful
0806 19:35:32.608092 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1180,"mutex_wait_us":456,"spinlock_wait_cycles":38656,"thread_start_us":1172,"threads_started":1}
I20260806 19:35:32.612452 17956 heartbeater.cc:347] Connected to a master server at 127.10.25.61:37363
I20260806 19:35:32.612862 17954 heartbeater.cc:347] Connected to a master server at 127.10.25.60:33869
I20260806 19:35:32.612874 17956 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:32.613034 17954 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:32.614130 17956 heartbeater.cc:511] Master 127.10.25.61:37363 requested a full tablet report, sending...
I20260806 19:35:32.614200 17954 heartbeater.cc:511] Master 127.10.25.60:33869 requested a full tablet report, sending...
I20260806 19:35:32.616353 17618 ts_manager.cc:194] Registered new tserver with Master: e787046186fe47b9bd9091b5a4c3e80d (127.10.25.2:35235)
I20260806 19:35:32.616358 17424 ts_manager.cc:194] Registered new tserver with Master: e787046186fe47b9bd9091b5a4c3e80d (127.10.25.2:35235)
I20260806 19:35:32.618948 17965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:32.569254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43496 (local address 127.10.25.3:45683)
0806 19:35:32.594101 (+ 24847us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:32.594114 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:32.594784 (+ 670us) server_negotiation.cc:407] Connection header received
0806 19:35:32.594999 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:32.595014 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:32.595303 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:32.595494 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:32.597263 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:32.598362 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:32.600659 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:32.601780 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:32.601915 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:32.609600 (+ 7685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:32.609616 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:32.609632 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:32.609679 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:32.614184 (+ 4505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:32.615323 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:32.615333 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:32.615343 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:32.615561 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:32.616573 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:32.616580 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:32.616583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:32.617187 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:32.617550 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:32.618143 (+ 593us) server_negotiation.cc:299] Negotiation successful
0806 19:35:32.618472 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24628,"thread_start_us":24464,"threads_started":1}
I20260806 19:35:32.620493 17271 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:32.620510 17963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:32.564163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:43203 (local address 127.10.25.62:39721)
0806 19:35:32.593026 (+ 28863us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:32.593036 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:32.593065 (+ 29us) server_negotiation.cc:407] Connection header received
0806 19:35:32.593138 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:32.593143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:32.593228 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:32.593352 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:32.595872 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:32.597006 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:32.599378 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:32.601101 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:32.601259 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:32.609713 (+ 8454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:32.609722 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:32.609727 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:32.609776 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:32.614794 (+ 5018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:32.616127 (+ 1333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:32.616133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:32.616137 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:32.616312 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:32.618764 (+ 2452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:32.618772 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:32.618775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:32.619211 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:32.619388 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:32.619823 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 19:35:32.620057 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28693,"thread_start_us":29181,"threads_started":1}
I20260806 19:35:32.620502 17962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:32.563999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.2:43203)
0806 19:35:32.589856 (+ 25857us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:32.589886 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:32.590047 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:32.594726 (+ 4679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:32.594733 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:32.595321 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:32.595684 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:32.595697 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:32.597610 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:32.597616 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:32.599202 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:32.599218 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:32.601642 (+ 2424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:32.601651 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:32.601873 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:32.602480 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:32.602504 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:32.607239 (+ 4735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:32.615310 (+ 8071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:32.615315 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:32.615318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:32.615918 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:32.616650 (+ 732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:32.616659 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:32.616663 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:32.617094 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:32.619363 (+ 2269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:32.619369 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:32.619557 (+ 188us) client_negotiation.cc:769] Sending connection context
0806 19:35:32.619713 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 19:35:32.620048 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25661,"thread_start_us":25410,"threads_started":1}
I20260806 19:35:32.622206 17955 heartbeater.cc:347] Connected to a master server at 127.10.25.62:39721
I20260806 19:35:32.622481 17955 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:32.623327 17955 heartbeater.cc:511] Master 127.10.25.62:39721 requested a full tablet report, sending...
I20260806 19:35:32.625084 16805 ts_manager.cc:194] Re-registered known tserver with Master: e787046186fe47b9bd9091b5a4c3e80d (127.10.25.2:35235)
I20260806 19:35:32.626633 16805 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:43203
I20260806 19:35:32.718406 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 17143
I20260806 19:35:32.744185 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:45683
--local_ip_for_outbound_sockets=127.10.25.3
--tserver_master_addrs=127.10.25.62:39721
--webserver_port=44201
--webserver_interface=127.10.25.3
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.3.keytab
--principal=oryx/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.25.62:39721,127.10.25.61:37363
--tserver_master_addrs=127.10.25.62:39721,127.10.25.61:37363,127.10.25.60:33869 with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:35:32.865643 17804 heartbeater.cc:503] Master 127.10.25.62:39721 was elected leader, sending a full tablet report...
W20260806 19:35:33.157305 17967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:33.157748 17967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:33.157859 17967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:33.166659 17967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:33.166868 17967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:33.167121 17967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:35:33.180051 17967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.3.keytab
--principal=oryx/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:45683
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=44201
--tserver_master_addrs=127.10.25.62:39721,127.10.25.61:37363,127.10.25.60:33869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:33.181623 17967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:33.183111 17967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:33.201579 17973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:33.201685 17972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:33.202524 17975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:33.203660 17967 server_base.cc:1034] running on GCE node
I20260806 19:35:33.204633 17967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:33.205853 17967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:33.207122 17967 hybrid_clock.cc:648] HybridClock initialized: now 1786044933207062 us; error 60 us; skew 500 ppm
I20260806 19:35:33.207487 17967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:33 dist-test-slave-ss4n krb5kdc[16758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044933, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:33.211372 17967 init.cc:377] Logged in from keytab as oryx/127.10.25.3@KRBTEST.COM (short username oryx)
I20260806 19:35:33.213286 17967 webserver.cc:460] Webserver started at http://127.10.25.3:44201/ using document root <none> and password file <none>
I20260806 19:35:33.214212 17967 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:33.214411 17967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:33.222076 17967 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 19:35:33.226003 17983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:33.228219 17967 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:35:33.228524 17967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "e90efef159644406821b0eab58afcf43"
format_stamp: "Formatted at 2026-08-06 19:35:28 on dist-test-slave-ss4n"
server_key: "92ecc404224a8e637e31043a25aa670d"
server_key_iv: "ed807a35fac411c132cc6047a7f8816f"
server_key_version: "encryptionkey@0"
I20260806 19:35:33.229861 17967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:33.254004 17967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:33.259873 17967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:33.260257 17967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:33.261435 17967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:33.263264 17967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:33.263407 17967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:33.263571 17967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:33.263674 17967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:33.306157 17967 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:45683
I20260806 19:35:33.306272 18099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:45683 every 8 connection(s)
I20260806 19:35:33.307878 17967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:35:33.315255 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 17967
Aug 06 19:35:33 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044933, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM
Aug 06 19:35:33 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044933, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.61@KRBTEST.COM
Aug 06 19:35:33 dist-test-slave-ss4n krb5kdc[16758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044933, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.60@KRBTEST.COM
I20260806 19:35:33.358003 18107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.312784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:55865 (local address 127.10.25.62:39721)
0806 19:35:33.328124 (+ 15340us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:33.328136 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:33.328167 (+ 31us) server_negotiation.cc:407] Connection header received
0806 19:35:33.328261 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:33.328270 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:33.328391 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:33.328550 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:33.331442 (+ 2892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.332625 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.337183 (+ 4558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.337909 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.338071 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.345309 (+ 7238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:33.345321 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:33.345329 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:33.345388 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:33.351265 (+ 5877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:33.353374 (+ 2109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:33.353381 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:33.353388 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:33.353672 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:33.354734 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:33.354740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:33.354745 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:33.355316 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:33.355516 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:33.356939 (+ 1423us) server_negotiation.cc:299] Negotiation successful
0806 19:35:33.357389 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15130,"thread_start_us":15883,"threads_started":1}
I20260806 19:35:33.358726 18104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.311392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:41813 (local address 127.10.25.60:33869)
0806 19:35:33.312730 (+ 1338us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:33.312738 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:33.329739 (+ 17001us) server_negotiation.cc:407] Connection header received
0806 19:35:33.330530 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:33.330540 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:33.330653 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:33.330805 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:33.332382 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.333878 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.337352 (+ 3474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.338071 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.338195 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.346161 (+ 7966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:33.346172 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:33.346178 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:33.346229 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:33.353911 (+ 7682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:33.355244 (+ 1333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:33.355250 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:33.355254 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:33.355509 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:33.356187 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:33.356196 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:33.356202 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:33.356706 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:33.356858 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:33.357772 (+ 914us) server_negotiation.cc:299] Negotiation successful
0806 19:35:33.358101 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1202,"thread_start_us":1088,"threads_started":1}
I20260806 19:35:33.361265 18109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.312298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:33869 (local address 127.10.25.3:41813)
0806 19:35:33.329600 (+ 17302us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:33.329633 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:33.330307 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:33.331102 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:33.331106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:33.331905 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:33.332198 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:33.332208 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.334098 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.334105 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:33.337167 (+ 3062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:33.337177 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.338510 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.338520 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:33.339249 (+ 729us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.340545 (+ 1296us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:33.340588 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:33.345885 (+ 5297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:33.354192 (+ 8307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:33.354201 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:33.354209 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:33.355042 (+ 833us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:33.355688 (+ 646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:33.355698 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:33.355705 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:33.355988 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:33.356940 (+ 952us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:33.356945 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:33.357119 (+ 174us) client_negotiation.cc:769] Sending connection context
0806 19:35:33.357406 (+ 287us) client_negotiation.cc:240] Negotiation successful
0806 19:35:33.359382 (+ 1976us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17098,"mutex_wait_us":1847,"spinlock_wait_cycles":52480,"thread_start_us":18017,"threads_started":1}
I20260806 19:35:33.364019 18108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.314825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:37561 (local address 127.10.25.61:37363)
0806 19:35:33.330174 (+ 15349us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:33.330180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:33.330970 (+ 790us) server_negotiation.cc:407] Connection header received
0806 19:35:33.331208 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:33.331217 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:33.331313 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:33.331488 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:33.332728 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.334087 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.337276 (+ 3189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.338112 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.338235 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.346035 (+ 7800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:33.346046 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:33.346051 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:33.346101 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:33.352343 (+ 6242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:33.353733 (+ 1390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:33.353743 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:33.353747 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:33.353999 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:33.360454 (+ 6455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:33.360465 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:33.360472 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:33.361038 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:33.361250 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:33.362592 (+ 1342us) server_negotiation.cc:299] Negotiation successful
0806 19:35:33.363007 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15203,"thread_start_us":15070,"threads_started":1}
I20260806 19:35:33.365768 18105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.312297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.3:55865)
0806 19:35:33.313590 (+ 1293us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:33.313660 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:33.323773 (+ 10113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:33.329050 (+ 5277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:33.329067 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:33.330139 (+ 1072us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:33.331238 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:33.331266 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.332769 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.332777 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:33.336956 (+ 4179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:33.336990 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.338365 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.338378 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:33.338880 (+ 502us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.340083 (+ 1203us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:33.340137 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:33.344885 (+ 4748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:33.351492 (+ 6607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:33.351510 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:33.351540 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:33.353138 (+ 1598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:33.354142 (+ 1004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:33.354151 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:33.354158 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:33.354586 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:33.355521 (+ 935us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:33.355550 (+ 29us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:33.356326 (+ 776us) client_negotiation.cc:769] Sending connection context
0806 19:35:33.356827 (+ 501us) client_negotiation.cc:240] Negotiation successful
0806 19:35:33.357919 (+ 1092us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":85,"thread_start_us":459,"threads_started":1}
I20260806 19:35:33.366760 18106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.315851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:37363 (local address 127.10.25.3:37561)
0806 19:35:33.330827 (+ 14976us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:33.330854 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:33.331046 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:33.331502 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:33.331510 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:33.332128 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:33.332529 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:33.332541 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.334277 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.334289 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:33.337084 (+ 2795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:33.337102 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.338659 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.338667 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:33.339109 (+ 442us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.340542 (+ 1433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:33.340567 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:33.345728 (+ 5161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:33.352620 (+ 6892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:33.352630 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:33.352637 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:33.353560 (+ 923us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:33.359892 (+ 6332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:33.359902 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:33.359910 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:33.360228 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:33.361919 (+ 1691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:33.361933 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:33.362160 (+ 227us) client_negotiation.cc:769] Sending connection context
0806 19:35:33.364744 (+ 2584us) client_negotiation.cc:240] Negotiation successful
0806 19:35:33.365272 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14848,"mutex_wait_us":160,"spinlock_wait_cycles":7040,"thread_start_us":13409,"threads_started":1}
I20260806 19:35:33.367862 18100 heartbeater.cc:347] Connected to a master server at 127.10.25.60:33869
I20260806 19:35:33.367929 18101 heartbeater.cc:347] Connected to a master server at 127.10.25.62:39721
I20260806 19:35:33.368266 18100 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:33.368983 18102 heartbeater.cc:347] Connected to a master server at 127.10.25.61:37363
I20260806 19:35:33.369149 18102 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:33.369536 18100 heartbeater.cc:511] Master 127.10.25.60:33869 requested a full tablet report, sending...
I20260806 19:35:33.369668 18102 heartbeater.cc:511] Master 127.10.25.61:37363 requested a full tablet report, sending...
I20260806 19:35:33.370009 18101 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:33.370556 18101 heartbeater.cc:511] Master 127.10.25.62:39721 requested a full tablet report, sending...
I20260806 19:35:33.372014 16805 ts_manager.cc:194] Re-registered known tserver with Master: e90efef159644406821b0eab58afcf43 (127.10.25.3:45683)
I20260806 19:35:33.373339 16805 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:55865
I20260806 19:35:33.375208 17424 ts_manager.cc:194] Registered new tserver with Master: e90efef159644406821b0eab58afcf43 (127.10.25.3:45683)
I20260806 19:35:33.379704 17618 ts_manager.cc:194] Registered new tserver with Master: e90efef159644406821b0eab58afcf43 (127.10.25.3:45683)
I20260806 19:35:33.390859 18107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.347346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59246 (local address 127.10.25.62:39721)
0806 19:35:33.359024 (+ 11678us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:33.359033 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:33.359057 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:35:33.359132 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:33.359140 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:33.359225 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:33.359335 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:33.365092 (+ 5757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.366893 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.373046 (+ 6153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.373929 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.374137 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.382842 (+ 8705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:33.382850 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:33.382855 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:33.382910 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:33.386695 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:33.388050 (+ 1355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:33.388058 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:33.388063 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:33.388263 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:33.388750 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:33.388756 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:33.388762 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:33.389258 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:33.389600 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:33.390088 (+ 488us) server_negotiation.cc:299] Negotiation successful
0806 19:35:33.390404 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11531,"thread_start_us":36360,"threads_started":1}
I20260806 19:35:33.393617 18104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.352003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39258 (local address 127.10.25.60:33869)
0806 19:35:33.359727 (+ 7724us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:33.359737 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:33.368692 (+ 8955us) server_negotiation.cc:407] Connection header received
0806 19:35:33.369092 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:33.369102 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:33.369280 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:33.369385 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:33.373554 (+ 4169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.375155 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.378780 (+ 3625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.380426 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.380595 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.385318 (+ 4723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:33.385326 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:33.385332 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:33.385383 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:33.389334 (+ 3951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:33.390378 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:33.390385 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:33.390388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:33.390572 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:33.391172 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:33.391179 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:33.391182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:33.391675 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:33.391949 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:33.392757 (+ 808us) server_negotiation.cc:299] Negotiation successful
0806 19:35:33.393094 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7563,"thread_start_us":15409,"threads_started":1}
I20260806 19:35:33.399700 18117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.348776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57262 (local address 127.10.25.61:37363)
0806 19:35:33.355330 (+ 6554us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:33.355340 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:33.355368 (+ 28us) server_negotiation.cc:407] Connection header received
0806 19:35:33.365573 (+ 10205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:33.365584 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:33.373307 (+ 7723us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:33.379701 (+ 6394us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:33.382834 (+ 3133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.384682 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.387058 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.387812 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.387975 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.391743 (+ 3768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:33.391752 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:33.391757 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:33.391804 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:33.396100 (+ 4296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:33.396919 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:33.396926 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:33.396930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:33.397098 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:33.397795 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:33.397801 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:33.397805 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:33.398193 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:33.398447 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:33.398992 (+ 545us) server_negotiation.cc:299] Negotiation successful
0806 19:35:33.399263 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6292,"thread_start_us":6111,"threads_started":1}
I20260806 19:35:33.396263 16805 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59246:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:35:33.405182 16805 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:35:33.464056 18135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.436521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.10.25.2:35235)
0806 19:35:33.450204 (+ 13683us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:33.450220 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:33.451940 (+ 1720us) server_negotiation.cc:407] Connection header received
0806 19:35:33.452111 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:33.452124 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:33.452513 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:33.452973 (+ 460us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:33.455511 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.457860 (+ 2349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.461245 (+ 3385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.462711 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.462898 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.463180 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:33.463535 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 19:35:33.463725 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13365,"thread_start_us":17140,"threads_started":1}
I20260806 19:35:33.464188 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.435993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:35235 (local address 127.0.0.1:56458)
0806 19:35:33.451364 (+ 15371us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:33.451396 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:33.451744 (+ 348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:33.453453 (+ 1709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:33.453463 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:33.453532 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:33.455294 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:33.455310 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.458170 (+ 2860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.458177 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:33.461031 (+ 2854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:33.461048 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.463012 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.463022 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:33.463310 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.463343 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 19:35:33.463471 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 19:35:33.463666 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15129,"mutex_wait_us":152,"thread_start_us":20490,"threads_started":1}
I20260806 19:35:33.465325 18133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.435602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:45683 (local address 127.0.0.1:43500)
0806 19:35:33.447490 (+ 11888us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:33.447524 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:33.447934 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:33.452355 (+ 4421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:33.452365 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:33.452411 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:33.455525 (+ 3114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:33.455544 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.458006 (+ 2462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.458013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:33.460578 (+ 2565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:33.460595 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.463664 (+ 3069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.463672 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:33.463945 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.463983 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 19:35:33.464089 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 19:35:33.464867 (+ 778us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9643,"mutex_wait_us":97,"thread_start_us":9356,"threads_started":1}
I20260806 19:35:33.467214 18137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.436949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43500 (local address 127.10.25.3:45683)
0806 19:35:33.450398 (+ 13449us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:33.450412 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:33.450503 (+ 91us) server_negotiation.cc:407] Connection header received
0806 19:35:33.450693 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:33.450708 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:33.451130 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:33.451698 (+ 568us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:33.455889 (+ 4191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.457833 (+ 1944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.460799 (+ 2966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.463377 (+ 2578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.465797 (+ 2420us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.466083 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:33.466167 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 19:35:33.466390 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13156,"thread_start_us":12904,"threads_started":1}
I20260806 19:35:33.468217 18134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.436627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:35055 (local address 127.0.0.1:49790)
0806 19:35:33.456029 (+ 19402us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:33.456053 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:33.456355 (+ 302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:33.458260 (+ 1905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:33.458268 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:33.458306 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:33.459101 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:33.459125 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.461999 (+ 2874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.462010 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:33.464992 (+ 2982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:33.465010 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.467194 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.467204 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:33.467426 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.467456 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 19:35:33.467590 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 19:35:33.467752 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19204,"mutex_wait_us":4,"thread_start_us":14167,"threads_started":1}
I20260806 19:35:33.468967 18138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.437869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49790 (local address 127.10.25.1:35055)
0806 19:35:33.457239 (+ 19370us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:33.457250 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:33.457284 (+ 34us) server_negotiation.cc:407] Connection header received
0806 19:35:33.457407 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:33.457418 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:33.457853 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:33.458265 (+ 412us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:33.459378 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.461474 (+ 2096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.465213 (+ 3739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.466802 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.467008 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.467304 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:33.467653 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 19:35:33.468680 (+ 1027us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19105,"thread_start_us":18974,"threads_started":1}
I20260806 19:35:33.472711 18031 tablet_service.cc:1467] Processing CreateTablet for tablet 3f7b94e82f684aa5b52580cc0d78e005 (DEFAULT_TABLE table=test-table [id=ac783a3bb93e47d592a166a607078151]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:33.473548 17736 tablet_service.cc:1467] Processing CreateTablet for tablet 3f7b94e82f684aa5b52580cc0d78e005 (DEFAULT_TABLE table=test-table [id=ac783a3bb93e47d592a166a607078151]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:33.474524 17885 tablet_service.cc:1467] Processing CreateTablet for tablet 3f7b94e82f684aa5b52580cc0d78e005 (DEFAULT_TABLE table=test-table [id=ac783a3bb93e47d592a166a607078151]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:33.474997 18031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f7b94e82f684aa5b52580cc0d78e005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:33.475564 17736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f7b94e82f684aa5b52580cc0d78e005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:33.476119 17885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f7b94e82f684aa5b52580cc0d78e005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:33.497002 18139 tablet_bootstrap.cc:492] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d: Bootstrap starting.
I20260806 19:35:33.500681 18140 tablet_bootstrap.cc:492] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: Bootstrap starting.
I20260806 19:35:33.502915 18139 tablet_bootstrap.cc:654] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:33.505178 18139 log.cc:824] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:33.505733 18140 tablet_bootstrap.cc:654] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:33.506850 18141 tablet_bootstrap.cc:492] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43: Bootstrap starting.
I20260806 19:35:33.507784 18140 log.cc:824] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:33.511121 18141 tablet_bootstrap.cc:654] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:33.512806 18141 log.cc:824] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:33.515229 18140 tablet_bootstrap.cc:492] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: No bootstrap required, opened a new log
I20260806 19:35:33.515623 18140 ts_tablet_manager.cc:1397] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.011s
I20260806 19:35:33.516151 18139 tablet_bootstrap.cc:492] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d: No bootstrap required, opened a new log
I20260806 19:35:33.516480 18139 ts_tablet_manager.cc:1397] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.008s
I20260806 19:35:33.521463 18141 tablet_bootstrap.cc:492] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43: No bootstrap required, opened a new log
I20260806 19:35:33.521874 18141 ts_tablet_manager.cc:1397] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.010s
I20260806 19:35:33.529685 18141 raft_consensus.cc:348] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:33.529731 18139 raft_consensus.cc:348] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:33.529685 18140 raft_consensus.cc:348] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:33.530210 18141 raft_consensus.cc:374] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:33.530236 18140 raft_consensus.cc:374] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:33.530257 18139 raft_consensus.cc:374] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:33.530329 18141 raft_consensus.cc:729] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90efef159644406821b0eab58afcf43, State: Initialized, Role: FOLLOWER
I20260806 19:35:33.530373 18139 raft_consensus.cc:729] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e787046186fe47b9bd9091b5a4c3e80d, State: Initialized, Role: FOLLOWER
I20260806 19:35:33.530360 18140 raft_consensus.cc:729] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dbd5f87907f42d9b9799cd6cdb1bf0c, State: Initialized, Role: FOLLOWER
I20260806 19:35:33.531109 18141 consensus_queue.cc:260] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:33.531128 18139 consensus_queue.cc:260] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:33.531131 18140 consensus_queue.cc:260] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:33.532706 18101 heartbeater.cc:503] Master 127.10.25.62:39721 was elected leader, sending a full tablet report...
I20260806 19:35:33.533563 18141 ts_tablet_manager.cc:1428] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43: Time spent starting tablet: real 0.012s user 0.010s sys 0.002s
I20260806 19:35:33.542418 18140 ts_tablet_manager.cc:1428] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: Time spent starting tablet: real 0.027s user 0.015s sys 0.005s
I20260806 19:35:33.542480 18139 ts_tablet_manager.cc:1428] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d: Time spent starting tablet: real 0.026s user 0.024s sys 0.000s
I20260806 19:35:33.544529 17955 heartbeater.cc:503] Master 127.10.25.62:39721 was elected leader, sending a full tablet report...
W20260806 19:35:33.560665 18103 tablet.cc:2369] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:35:33.561233 17806 tablet.cc:2369] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:35:33.561937 17957 tablet.cc:2369] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:35:33.685928 18146 raft_consensus.cc:482] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:35:33.686589 18146 raft_consensus.cc:504] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:33.688683 18146 leader_election.cc:288] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90efef159644406821b0eab58afcf43 (127.10.25.3:45683), 3dbd5f87907f42d9b9799cd6cdb1bf0c (127.10.25.1:35055)
I20260806 19:35:33.701838 18138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.689628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:60227 (local address 127.10.25.1:35055)
0806 19:35:33.690035 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:33.690045 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:33.690794 (+ 749us) server_negotiation.cc:407] Connection header received
0806 19:35:33.690903 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:33.690909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:33.691071 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:33.691174 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:33.692492 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.693907 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.697986 (+ 4079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.699815 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.700100 (+ 285us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.700392 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:33.701068 (+ 676us) server_negotiation.cc:299] Negotiation successful
0806 19:35:33.701296 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260806 19:35:33.701906 18137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.689180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:47965 (local address 127.10.25.3:45683)
0806 19:35:33.689477 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:33.689516 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:33.690529 (+ 1013us) server_negotiation.cc:407] Connection header received
0806 19:35:33.690867 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:33.690873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:33.690988 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:33.691136 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:33.692492 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.693907 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.697986 (+ 4079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.699841 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.700100 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.700396 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:33.701068 (+ 672us) server_negotiation.cc:299] Negotiation successful
0806 19:35:33.701296 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260806 19:35:33.702394 18148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.689008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:45683 (local address 127.10.25.2:47965)
0806 19:35:33.690194 (+ 1186us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:33.690221 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:33.690618 (+ 397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:33.691184 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:33.691189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:33.691235 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:33.692271 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:33.692290 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.694130 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.694137 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:33.697557 (+ 3420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:33.697585 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.700393 (+ 2808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.700403 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:33.700748 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.700819 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 19:35:33.700973 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 19:35:33.701182 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"mutex_wait_us":358,"spinlock_wait_cycles":4480,"thread_start_us":737,"threads_started":1}
I20260806 19:35:33.702450 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.689217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:35055 (local address 127.10.25.2:60227)
0806 19:35:33.690536 (+ 1319us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:33.690559 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:33.690791 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:33.691184 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:33.691189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:33.691228 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:33.692271 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:33.692290 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.694138 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.694143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:33.697557 (+ 3414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:33.697585 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.700408 (+ 2823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.700415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:33.700730 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.700818 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 19:35:33.700973 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 19:35:33.701182 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1175,"mutex_wait_us":309,"spinlock_wait_cycles":1408,"thread_start_us":909,"threads_started":1}
I20260806 19:35:33.703889 18054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7b94e82f684aa5b52580cc0d78e005" candidate_uuid: "e787046186fe47b9bd9091b5a4c3e80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90efef159644406821b0eab58afcf43" is_pre_election: true
I20260806 19:35:33.704484 18054 raft_consensus.cc:2454] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e787046186fe47b9bd9091b5a4c3e80d in term 0.
I20260806 19:35:33.704324 17756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7b94e82f684aa5b52580cc0d78e005" candidate_uuid: "e787046186fe47b9bd9091b5a4c3e80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" is_pre_election: true
I20260806 19:35:33.704998 17756 raft_consensus.cc:2454] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e787046186fe47b9bd9091b5a4c3e80d in term 0.
I20260806 19:35:33.705402 17839 leader_election.cc:302] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e787046186fe47b9bd9091b5a4c3e80d, e90efef159644406821b0eab58afcf43; no voters:
I20260806 19:35:33.706058 18146 raft_consensus.cc:2781] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:35:33.706238 18146 raft_consensus.cc:482] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:35:33.706350 18146 raft_consensus.cc:3037] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:33.709924 18146 raft_consensus.cc:504] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:33.710844 18146 leader_election.cc:288] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [CANDIDATE]: Term 1 election: Requested vote from peers e90efef159644406821b0eab58afcf43 (127.10.25.3:45683), 3dbd5f87907f42d9b9799cd6cdb1bf0c (127.10.25.1:35055)
I20260806 19:35:33.711516 18054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7b94e82f684aa5b52580cc0d78e005" candidate_uuid: "e787046186fe47b9bd9091b5a4c3e80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90efef159644406821b0eab58afcf43"
I20260806 19:35:33.711555 17756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7b94e82f684aa5b52580cc0d78e005" candidate_uuid: "e787046186fe47b9bd9091b5a4c3e80d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c"
I20260806 19:35:33.711812 18054 raft_consensus.cc:3037] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:33.711817 17756 raft_consensus.cc:3037] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:33.714741 18054 raft_consensus.cc:2454] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e787046186fe47b9bd9091b5a4c3e80d in term 1.
I20260806 19:35:33.715493 17839 leader_election.cc:302] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e787046186fe47b9bd9091b5a4c3e80d, e90efef159644406821b0eab58afcf43; no voters:
I20260806 19:35:33.716045 18146 raft_consensus.cc:2781] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:33.716318 17756 raft_consensus.cc:2454] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e787046186fe47b9bd9091b5a4c3e80d in term 1.
I20260806 19:35:33.717332 18146 raft_consensus.cc:686] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [term 1 LEADER]: Becoming Leader. State: Replica: e787046186fe47b9bd9091b5a4c3e80d, State: Running, Role: LEADER
I20260806 19:35:33.718250 18146 consensus_queue.cc:237] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:33.724777 16805 catalog_manager.cc:5495] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d reported cstate change: term changed from 0 to 1, leader changed from <none> to e787046186fe47b9bd9091b5a4c3e80d (127.10.25.2). New cstate: current_term: 1 leader_uuid: "e787046186fe47b9bd9091b5a4c3e80d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } health_report { overall_health: UNKNOWN } } }
I20260806 19:35:33.756639 18135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.748370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56466 (local address 127.10.25.2:35235)
0806 19:35:33.748626 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:33.748632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:33.748658 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:35:33.748813 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:33.748817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:33.748949 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:33.749159 (+ 210us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:35:33.750229 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.751353 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.753455 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.754188 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.754346 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.755027 (+ 681us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:35:33.755649 (+ 622us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:35:33.755770 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:33.756165 (+ 395us) server_negotiation.cc:299] Negotiation successful
0806 19:35:33.756256 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 19:35:33.769630 17756 raft_consensus.cc:1261] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 1 FOLLOWER]: Refusing update from remote peer e787046186fe47b9bd9091b5a4c3e80d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:35:33.769601 18054 raft_consensus.cc:1261] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43 [term 1 FOLLOWER]: Refusing update from remote peer e787046186fe47b9bd9091b5a4c3e80d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:35:33.771306 18150 consensus_queue.cc:1035] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:33.772059 18146 consensus_queue.cc:1035] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:33.814492 16806 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:35:33.815012 16806 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:35:33.825091 16806 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad1e547ad04f471d9301eae45ffcec91: Sending DeleteTablet for 3 replicas of tablet 3f7b94e82f684aa5b52580cc0d78e005
I20260806 19:35:33.827450 17736 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f7b94e82f684aa5b52580cc0d78e005 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:33 UTC) from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:49790
I20260806 19:35:33.827379 17885 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f7b94e82f684aa5b52580cc0d78e005 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:33 UTC) from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:56458
I20260806 19:35:33.829306 18031 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f7b94e82f684aa5b52580cc0d78e005 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:33 UTC) from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:43500
I20260806 19:35:33.841099 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 16772
I20260806 19:35:33.841447 18163 tablet_replica.cc:330] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d: stopping tablet replica
I20260806 19:35:33.842566 18163 raft_consensus.cc:2229] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:35:33.843720 16837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:33.843722 18163 raft_consensus.cc:2258] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:33.845563 18165 tablet_replica.cc:330] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43: stopping tablet replica
I20260806 19:35:33.846302 18165 raft_consensus.cc:2229] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:33.847205 18165 raft_consensus.cc:2258] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:33.848536 18164 tablet_replica.cc:330] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: stopping tablet replica
I20260806 19:35:33.849098 18164 raft_consensus.cc:2229] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:33.849833 18164 raft_consensus.cc:2258] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:33.852468 18165 ts_tablet_manager.cc:1905] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:33.852500 18163 ts_tablet_manager.cc:1905] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:33.862268 18163 ts_tablet_manager.cc:1918] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:33.862576 18163 log.cc:1196] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/3f7b94e82f684aa5b52580cc0d78e005
I20260806 19:35:33.863422 18163 ts_tablet_manager.cc:1939] T 3f7b94e82f684aa5b52580cc0d78e005 P e787046186fe47b9bd9091b5a4c3e80d: Deleting consensus metadata
I20260806 19:35:33.859248 18164 ts_tablet_manager.cc:1905] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:33.873365 18117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.830483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57266 (local address 127.10.25.61:37363)
0806 19:35:33.830724 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:33.830734 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:33.838496 (+ 7762us) server_negotiation.cc:407] Connection header received
0806 19:35:33.840029 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:33.840040 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:33.840527 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:33.840770 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:33.845844 (+ 5074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.847408 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.858024 (+ 10616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.858965 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.859117 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.863857 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:33.863865 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:33.863871 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:33.863926 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:33.868405 (+ 4479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:33.869600 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:33.869609 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:33.869616 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:33.869822 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:33.870420 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:33.870425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:33.870427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:33.870989 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:33.871428 (+ 439us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:33.872374 (+ 946us) server_negotiation.cc:299] Negotiation successful
0806 19:35:33.872796 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 19:35:33.876078 18164 ts_tablet_manager.cc:1918] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:33.876308 18164 log.cc:1196] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/3f7b94e82f684aa5b52580cc0d78e005
I20260806 19:35:33.877234 18164 ts_tablet_manager.cc:1939] T 3f7b94e82f684aa5b52580cc0d78e005 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: Deleting consensus metadata
I20260806 19:35:33.884961 18165 ts_tablet_manager.cc:1918] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:33.885233 18165 log.cc:1196] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/3f7b94e82f684aa5b52580cc0d78e005
I20260806 19:35:33.886264 18165 ts_tablet_manager.cc:1939] T 3f7b94e82f684aa5b52580cc0d78e005 P e90efef159644406821b0eab58afcf43: Deleting consensus metadata
I20260806 19:35:33.902091 18104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:33.838785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39268 (local address 127.10.25.60:33869)
0806 19:35:33.839121 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:33.839130 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:33.877986 (+ 38856us) server_negotiation.cc:407] Connection header received
0806 19:35:33.878307 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:33.878314 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:33.878414 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:33.878499 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:33.881826 (+ 3327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.882655 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.884969 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:33.886000 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:33.886200 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:33.890575 (+ 4375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:33.890582 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:33.890586 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:33.890682 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:33.895230 (+ 4548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:33.896148 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:33.896155 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:33.896165 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:33.896328 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:33.897111 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:33.897117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:33.897119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:33.897540 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:33.898264 (+ 724us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:33.900728 (+ 2464us) server_negotiation.cc:299] Negotiation successful
0806 19:35:33.901036 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260806 19:35:34.062379 16789 catalog_manager.cc:4841] TS e787046186fe47b9bd9091b5a4c3e80d (127.10.25.2:35235): tablet 3f7b94e82f684aa5b52580cc0d78e005 (table test-table [id=ac783a3bb93e47d592a166a607078151]) successfully deleted
I20260806 19:35:34.062503 16790 catalog_manager.cc:4841] TS e90efef159644406821b0eab58afcf43 (127.10.25.3:45683): tablet 3f7b94e82f684aa5b52580cc0d78e005 (table test-table [id=ac783a3bb93e47d592a166a607078151]) successfully deleted
I20260806 19:35:34.063545 16791 catalog_manager.cc:4841] TS 3dbd5f87907f42d9b9799cd6cdb1bf0c (127.10.25.1:35055): tablet 3f7b94e82f684aa5b52580cc0d78e005 (table test-table [id=ac783a3bb93e47d592a166a607078151]) successfully deleted
I20260806 19:35:34.063956 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 16772
I20260806 19:35:34.131680 18117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:34.112560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57272 (local address 127.10.25.61:37363)
0806 19:35:34.112753 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:34.112760 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:34.114143 (+ 1383us) server_negotiation.cc:407] Connection header received
0806 19:35:34.114219 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:34.114225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:34.114309 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:34.114424 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:34.116801 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:34.118331 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:34.120291 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:34.120780 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:34.120950 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:34.124742 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:34.124748 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:34.124752 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:34.124795 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:34.128256 (+ 3461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:34.129068 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:34.129077 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:34.129083 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:34.129255 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:34.129767 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:34.129774 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:34.129778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:34.130105 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:34.130388 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:34.130817 (+ 429us) server_negotiation.cc:299] Negotiation successful
0806 19:35:34.131112 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 19:35:34.134424 18104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:34.111442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39280 (local address 127.10.25.60:33869)
0806 19:35:34.111691 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:34.111699 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:34.113398 (+ 1699us) server_negotiation.cc:407] Connection header received
0806 19:35:34.113766 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:34.113775 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:34.113872 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:34.113981 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:34.117330 (+ 3349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:34.119026 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:34.121319 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:34.122101 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:34.122283 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:34.126327 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:34.126336 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:34.126341 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:34.126395 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:34.130535 (+ 4140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:34.131580 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:34.131585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:34.131589 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:34.131757 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:34.132295 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:34.132301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:34.132305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:34.132638 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:34.132926 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:34.133801 (+ 875us) server_negotiation.cc:299] Negotiation successful
0806 19:35:34.134079 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 19:35:34.797596 18175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:34.796117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.2:42877)
0806 19:35:34.796922 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:34.797088 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":488,"threads_started":1}
W20260806 19:35:34.798408 17955 heartbeater.cc:644] Failed to heartbeat to 127.10.25.62:39721 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
I20260806 19:35:34.805840 18176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:34.803805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.3:48993)
0806 19:35:34.805377 (+ 1572us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:34.805574 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1442,"thread_start_us":1337,"threads_started":1}
W20260806 19:35:34.806511 18101 heartbeater.cc:644] Failed to heartbeat to 127.10.25.62:39721 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
I20260806 19:35:34.808023 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:34.807119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.1:47223)
0806 19:35:34.807647 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:34.807821 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":257,"threads_started":1}
W20260806 19:35:34.808391 17804 heartbeater.cc:644] Failed to heartbeat to 127.10.25.62:39721 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
I20260806 19:35:35.326465 18178 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad1e547ad04f471d9301eae45ffcec91)
I20260806 19:35:35.326538 18179 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad1e547ad04f471d9301eae45ffcec91)
I20260806 19:35:35.326853 18179 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } }
I20260806 19:35:35.326942 18178 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } }
I20260806 19:35:35.328241 18178 leader_election.cc:288] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad1e547ad04f471d9301eae45ffcec91 (127.10.25.62:39721), 9c4294fa83ab48fda6e92f76562d93b5 (127.10.25.60:33869)
I20260806 19:35:35.329782 18180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.328464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:59678)
0806 19:35:35.329156 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.329411 (+ 255us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":337,"threads_started":1}
I20260806 19:35:35.330113 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.328608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:59684)
0806 19:35:35.329556 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.329793 (+ 237us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":787,"thread_start_us":621,"threads_started":1}
W20260806 19:35:35.330145 17409 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
I20260806 19:35:35.330878 18179 leader_election.cc:288] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad1e547ad04f471d9301eae45ffcec91 (127.10.25.62:39721), cdefd62fa4344bdba4e972b3c82124e5 (127.10.25.61:37363)
W20260806 19:35:35.331516 17602 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
I20260806 19:35:35.344096 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.343329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:59686)
0806 19:35:35.343558 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.343790 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 19:35:35.344372 17602 leader_election.cc:334] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ad1e547ad04f471d9301eae45ffcec91 (127.10.25.62:39721): Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
I20260806 19:35:35.344964 18187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.344186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:59698)
0806 19:35:35.344495 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.344724 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
W20260806 19:35:35.345216 17409 leader_election.cc:334] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ad1e547ad04f471d9301eae45ffcec91 (127.10.25.62:39721): Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
I20260806 19:35:35.347517 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.329893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:37363 (local address 127.0.0.1:57282)
0806 19:35:35.330403 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.330435 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:35.330630 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:35.336770 (+ 6140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:35.336780 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:35.336836 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:35.337214 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:35.337233 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.342911 (+ 5678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.342920 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:35.345170 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:35.345191 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.346636 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.346642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:35.346823 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:35.346849 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 19:35:35.346963 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 19:35:35.347121 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":57,"thread_start_us":1316,"threads_started":1}
I20260806 19:35:35.347540 18185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.330230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57282 (local address 127.10.25.61:37363)
0806 19:35:35.334478 (+ 4248us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:35.334489 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:35.334528 (+ 39us) server_negotiation.cc:407] Connection header received
0806 19:35:35.334653 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:35.334663 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:35.334777 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:35.337541 (+ 2764us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:35.337897 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.339336 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.345393 (+ 6057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.346437 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.346603 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:35.346703 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:35.346992 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 19:35:35.347148 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":4064,"thread_start_us":3778,"threads_started":1}
I20260806 19:35:35.348426 17433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c4294fa83ab48fda6e92f76562d93b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cdefd62fa4344bdba4e972b3c82124e5" is_pre_election: true
I20260806 19:35:35.349061 17433 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c4294fa83ab48fda6e92f76562d93b5 in term 1.
I20260806 19:35:35.349757 17602 leader_election.cc:302] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c4294fa83ab48fda6e92f76562d93b5, cdefd62fa4344bdba4e972b3c82124e5; no voters: ad1e547ad04f471d9301eae45ffcec91
I20260806 19:35:35.350070 18180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.331545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:33869 (local address 127.0.0.1:39284)
0806 19:35:35.336294 (+ 4749us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.336342 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:35.336551 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:35.342375 (+ 5824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:35.342382 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:35.342427 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:35.342942 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:35.342960 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.345547 (+ 2587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.345555 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:35.347556 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:35.347583 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.349130 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.349137 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:35.349334 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:35.349374 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 19:35:35.349533 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 19:35:35.349705 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4575,"mutex_wait_us":1,"thread_start_us":11007,"threads_started":1}
I20260806 19:35:35.350194 18182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.329032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39284 (local address 127.10.25.60:33869)
0806 19:35:35.341885 (+ 12853us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:35.341895 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:35.341921 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:35:35.342021 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:35.342030 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:35.342145 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:35.342330 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:35.343196 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.345369 (+ 2173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.347886 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.348910 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.349052 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:35.349184 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:35.349617 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 19:35:35.349757 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":436,"server-negotiator.queue_time_us":12640,"thread_start_us":12511,"threads_started":1}
I20260806 19:35:35.350247 18179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 19:35:35.350430 18179 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad1e547ad04f471d9301eae45ffcec91)
I20260806 19:35:35.350572 18179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 1 FOLLOWER]: Advancing to term 2
I20260806 19:35:35.350936 17628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cdefd62fa4344bdba4e972b3c82124e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9c4294fa83ab48fda6e92f76562d93b5" is_pre_election: true
I20260806 19:35:35.353201 18179 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } }
I20260806 19:35:35.353785 17628 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cdefd62fa4344bdba4e972b3c82124e5 in current term 2: Already voted for candidate 9c4294fa83ab48fda6e92f76562d93b5 in this term.
I20260806 19:35:35.354251 18179 leader_election.cc:288] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [CANDIDATE]: Term 2 election: Requested vote from peers ad1e547ad04f471d9301eae45ffcec91 (127.10.25.62:39721), cdefd62fa4344bdba4e972b3c82124e5 (127.10.25.61:37363)
I20260806 19:35:35.355181 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.354538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:59702)
0806 19:35:35.354770 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.354896 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 19:35:35.355356 17411 leader_election.cc:302] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cdefd62fa4344bdba4e972b3c82124e5; no voters: 9c4294fa83ab48fda6e92f76562d93b5, ad1e547ad04f471d9301eae45ffcec91
I20260806 19:35:35.355581 17433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c4294fa83ab48fda6e92f76562d93b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cdefd62fa4344bdba4e972b3c82124e5"
I20260806 19:35:35.355824 18178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 1 FOLLOWER]: Advancing to term 2
I20260806 19:35:35.357070 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.356597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:59706)
0806 19:35:35.356826 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.356917 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 19:35:35.357260 17602 leader_election.cc:334] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ad1e547ad04f471d9301eae45ffcec91 (127.10.25.62:39721): Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
I20260806 19:35:35.358814 18178 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 19:35:35.362790 17433 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c4294fa83ab48fda6e92f76562d93b5 in term 2.
I20260806 19:35:35.363857 17602 leader_election.cc:302] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9c4294fa83ab48fda6e92f76562d93b5, cdefd62fa4344bdba4e972b3c82124e5; no voters: ad1e547ad04f471d9301eae45ffcec91
I20260806 19:35:35.364601 18179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 19:35:35.365315 18179 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [term 2 LEADER]: Becoming Leader. State: Replica: 9c4294fa83ab48fda6e92f76562d93b5, State: Running, Role: LEADER
I20260806 19:35:35.365809 18179 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } }
I20260806 19:35:35.368064 18189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c4294fa83ab48fda6e92f76562d93b5. Latest consensus state: current_term: 2 leader_uuid: "9c4294fa83ab48fda6e92f76562d93b5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } } }
I20260806 19:35:35.368413 18189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:35.369103 18191 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:35:35.374142 18191 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:35:35.375226 18191 catalog_manager.cc:1259] Loaded cluster ID: f3614b79fba74957ae460f324c0676da
I20260806 19:35:35.375361 18191 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:35:35.376765 18191 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:35:35.377844 18191 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: Loaded TSK: 0
I20260806 19:35:35.378897 18191 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:35:35.424945 17618 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:35:35.432991 17618 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:35:35.438449 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.437950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:59740)
0806 19:35:35.438148 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.438241 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 19:35:35.438920 17433 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [term 2 FOLLOWER]: Refusing update from remote peer 9c4294fa83ab48fda6e92f76562d93b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 19:35:35.439730 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.439291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:59744)
0806 19:35:35.439479 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.439564 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 19:35:35.439813 18189 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 19:35:35.440089 17602 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 -> Peer ad1e547ad04f471d9301eae45ffcec91 (127.10.25.62:39721): Couldn't send request to peer ad1e547ad04f471d9301eae45ffcec91. Status: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 19:35:35.443356 18178 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c4294fa83ab48fda6e92f76562d93b5. Latest consensus state: current_term: 2 leader_uuid: "9c4294fa83ab48fda6e92f76562d93b5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } } }
I20260806 19:35:35.443747 18178 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:35:35.446678 18189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9c4294fa83ab48fda6e92f76562d93b5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } } }
I20260806 19:35:35.447067 18189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5 [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:35.448416 18178 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9c4294fa83ab48fda6e92f76562d93b5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad1e547ad04f471d9301eae45ffcec91" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39721 } } peers { permanent_uuid: "cdefd62fa4344bdba4e972b3c82124e5" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 37363 } attrs { promote: false } } peers { permanent_uuid: "9c4294fa83ab48fda6e92f76562d93b5" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 33869 } attrs { promote: false } } }
I20260806 19:35:35.448899 18178 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdefd62fa4344bdba4e972b3c82124e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:35:35.464701 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.457861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:35235 (local address 127.0.0.1:56474)
0806 19:35:35.458396 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.458419 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:35.458542 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:35.458933 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:35.458938 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:35.458963 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:35.459267 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:35.459277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.460689 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.460695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:35.462507 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:35.462522 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.463812 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.463819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:35.464024 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:35.464053 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 19:35:35.464170 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 19:35:35.464340 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373}
I20260806 19:35:35.464749 18198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.458055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56474 (local address 127.10.25.2:35235)
0806 19:35:35.458560 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:35.458566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:35.458588 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:35:35.458670 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:35.458675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:35.458772 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:35.458901 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:35.459418 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.460468 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.462687 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.463562 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.463735 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:35.463887 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:35.464212 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 19:35:35.464370 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":574,"threads_started":1}
I20260806 19:35:35.466027 17885 tablet_service.cc:1467] Processing CreateTablet for tablet 27ca366ed45b4abc9617ec06055006e7 (DEFAULT_TABLE table=test-table [id=a2b5a69f3ebd4dae8cbe85bf31b59676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:35.466548 17885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27ca366ed45b4abc9617ec06055006e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:35.473474 18201 tablet_bootstrap.cc:492] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d: Bootstrap starting.
I20260806 19:35:35.475968 18201 tablet_bootstrap.cc:654] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:35.478116 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.457250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:45683 (local address 127.0.0.1:43506)
0806 19:35:35.457473 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.457528 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:35.457651 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:35.470817 (+ 13166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:35.470824 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:35.470862 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:35.471080 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:35.471092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.472483 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.472492 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:35.474778 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:35.474796 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.477049 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.477057 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:35.477328 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:35.477368 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 19:35:35.477554 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 19:35:35.477732 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":1}
I20260806 19:35:35.478142 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.457683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43506 (local address 127.10.25.3:45683)
0806 19:35:35.470462 (+ 12779us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:35.470471 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:35.470501 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:35:35.470580 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:35.470584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:35.470646 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:35.470785 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:35.471255 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.472327 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.475064 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.476642 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.476808 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:35.476936 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:35.477580 (+ 644us) server_negotiation.cc:299] Negotiation successful
0806 19:35:35.477746 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12626,"thread_start_us":13228,"threads_started":1}
I20260806 19:35:35.479054 18031 tablet_service.cc:1467] Processing CreateTablet for tablet 27ca366ed45b4abc9617ec06055006e7 (DEFAULT_TABLE table=test-table [id=a2b5a69f3ebd4dae8cbe85bf31b59676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:35.479354 18201 tablet_bootstrap.cc:492] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d: No bootstrap required, opened a new log
I20260806 19:35:35.479521 18201 ts_tablet_manager.cc:1397] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 19:35:35.479573 18031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27ca366ed45b4abc9617ec06055006e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:35.480126 18200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.460682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49796 (local address 127.10.25.1:35055)
0806 19:35:35.469968 (+ 9286us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:35.469978 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:35.470006 (+ 28us) server_negotiation.cc:407] Connection header received
0806 19:35:35.470092 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:35.470098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:35.470185 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:35.470314 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:35.473741 (+ 3427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.474827 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.476736 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.479082 (+ 2346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.479250 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:35.479360 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:35.479643 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 19:35:35.479788 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9131,"thread_start_us":8987,"threads_started":1}
I20260806 19:35:35.480121 18199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.460083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:35055 (local address 127.0.0.1:49796)
0806 19:35:35.460850 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.460871 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:35.461024 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:35.473031 (+ 12007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:35.473036 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:35.473068 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:35.473397 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:35.473405 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.474962 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.474969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:35.476408 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:35.476423 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.479290 (+ 2867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.479296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:35.479473 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:35.479497 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 19:35:35.479613 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 19:35:35.479761 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":750,"threads_started":1}
I20260806 19:35:35.480506 18201 raft_consensus.cc:348] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:35.480793 18201 raft_consensus.cc:374] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:35.480898 18201 raft_consensus.cc:729] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e787046186fe47b9bd9091b5a4c3e80d, State: Initialized, Role: FOLLOWER
I20260806 19:35:35.481187 18201 consensus_queue.cc:260] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:35.482192 18201 ts_tablet_manager.cc:1428] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 19:35:35.486126 17954 heartbeater.cc:503] Master 127.10.25.60:33869 was elected leader, sending a full tablet report...
I20260806 19:35:35.488336 17736 tablet_service.cc:1467] Processing CreateTablet for tablet 27ca366ed45b4abc9617ec06055006e7 (DEFAULT_TABLE table=test-table [id=a2b5a69f3ebd4dae8cbe85bf31b59676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:35.489064 17736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27ca366ed45b4abc9617ec06055006e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:35.489913 18205 tablet_bootstrap.cc:492] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43: Bootstrap starting.
I20260806 19:35:35.492710 18204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.482658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.2:49797)
0806 19:35:35.490893 (+ 8235us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.492273 (+ 1380us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8062,"thread_start_us":7879,"threads_started":1}
I20260806 19:35:35.494094 18204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.493657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.2:56705)
0806 19:35:35.493849 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.493948 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 19:35:35.494109 18205 tablet_bootstrap.cc:654] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:35.496987 18205 tablet_bootstrap.cc:492] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43: No bootstrap required, opened a new log
I20260806 19:35:35.497336 18205 ts_tablet_manager.cc:1397] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260806 19:35:35.498631 18207 tablet_bootstrap.cc:492] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: Bootstrap starting.
I20260806 19:35:35.498620 18205 raft_consensus.cc:348] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:35.498925 18205 raft_consensus.cc:374] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:35.499027 18205 raft_consensus.cc:729] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90efef159644406821b0eab58afcf43, State: Initialized, Role: FOLLOWER
I20260806 19:35:35.499310 18205 consensus_queue.cc:260] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:35.500289 18205 ts_tablet_manager.cc:1428] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 19:35:35.500418 18207 tablet_bootstrap.cc:654] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:35.502638 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.501222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.3:35719)
0806 19:35:35.501932 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.502097 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":424,"threads_started":1}
I20260806 19:35:35.503557 18100 heartbeater.cc:503] Master 127.10.25.60:33869 was elected leader, sending a full tablet report...
I20260806 19:35:35.504035 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.503403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.3:44579)
0806 19:35:35.503666 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.503802 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 19:35:35.507103 18207 tablet_bootstrap.cc:492] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: No bootstrap required, opened a new log
I20260806 19:35:35.507329 18207 ts_tablet_manager.cc:1397] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 19:35:35.508078 18207 raft_consensus.cc:348] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:35.508333 18207 raft_consensus.cc:374] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:35.508462 18207 raft_consensus.cc:729] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dbd5f87907f42d9b9799cd6cdb1bf0c, State: Initialized, Role: FOLLOWER
I20260806 19:35:35.508744 18207 consensus_queue.cc:260] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:35.509660 18207 ts_tablet_manager.cc:1428] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 19:35:35.511346 18212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.510055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.1:60585)
0806 19:35:35.510838 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.511024 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":422,"threads_started":1}
I20260806 19:35:35.512328 17803 heartbeater.cc:503] Master 127.10.25.60:33869 was elected leader, sending a full tablet report...
I20260806 19:35:35.512791 18212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.512296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.1:53915)
0806 19:35:35.512493 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.512624 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 19:35:35.711908 18211 raft_consensus.cc:482] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:35:35.712849 18211 raft_consensus.cc:504] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:35.715543 18211 leader_election.cc:288] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e90efef159644406821b0eab58afcf43 (127.10.25.3:45683), e787046186fe47b9bd9091b5a4c3e80d (127.10.25.2:35235)
I20260806 19:35:35.725076 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.716358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:35365 (local address 127.10.25.3:45683)
0806 19:35:35.716623 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:35.716631 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:35.716661 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:35:35.716990 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:35.717000 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:35.717097 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:35.717218 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:35.717945 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.719270 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.722604 (+ 3334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.723855 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.724066 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:35.724198 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:35.724597 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 19:35:35.724759 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 19:35:35.725139 18212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.715827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:45683 (local address 127.10.25.1:35365)
0806 19:35:35.716341 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.716371 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:35.716800 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:35.717262 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:35.717271 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:35.717308 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:35.717717 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:35.717731 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.719492 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.719498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:35.722285 (+ 2787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:35.722311 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.724149 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.724167 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:35.724375 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:35.724406 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 19:35:35.724552 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 19:35:35.724740 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345}
I20260806 19:35:35.725926 18198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.716371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:41321 (local address 127.10.25.2:35235)
0806 19:35:35.716625 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:35.716634 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:35.717333 (+ 699us) server_negotiation.cc:407] Connection header received
0806 19:35:35.717806 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:35.717815 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:35.718007 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:35.718207 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:35.718790 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.720135 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.723427 (+ 3292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.724481 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.724713 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:35.724863 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:35.725370 (+ 507us) server_negotiation.cc:299] Negotiation successful
0806 19:35:35.725593 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 19:35:35.725924 18054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27ca366ed45b4abc9617ec06055006e7" candidate_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90efef159644406821b0eab58afcf43" is_pre_election: true
I20260806 19:35:35.726228 18054 raft_consensus.cc:2454] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3dbd5f87907f42d9b9799cd6cdb1bf0c in term 0.
I20260806 19:35:35.726848 18213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.716154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:35235 (local address 127.10.25.1:41321)
0806 19:35:35.717231 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.717252 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:35.717510 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:35.718200 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:35.718205 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:35.718231 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:35.718594 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:35.718607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.720306 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.720315 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:35.723024 (+ 2709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:35.723064 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.724859 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.724868 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:35.725091 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:35.725128 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 19:35:35.726306 (+ 1178us) client_negotiation.cc:240] Negotiation successful
0806 19:35:35.726518 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"thread_start_us":1600,"threads_started":1}
I20260806 19:35:35.726944 17689 leader_election.cc:302] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dbd5f87907f42d9b9799cd6cdb1bf0c, e90efef159644406821b0eab58afcf43; no voters:
I20260806 19:35:35.727595 18211 raft_consensus.cc:2781] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:35:35.727748 18211 raft_consensus.cc:482] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:35:35.728075 18211 raft_consensus.cc:3037] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:35.728180 17906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27ca366ed45b4abc9617ec06055006e7" candidate_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e787046186fe47b9bd9091b5a4c3e80d" is_pre_election: true
I20260806 19:35:35.728650 17906 raft_consensus.cc:2454] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3dbd5f87907f42d9b9799cd6cdb1bf0c in term 0.
I20260806 19:35:35.731051 18211 raft_consensus.cc:504] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:35.731781 18211 leader_election.cc:288] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [CANDIDATE]: Term 1 election: Requested vote from peers e90efef159644406821b0eab58afcf43 (127.10.25.3:45683), e787046186fe47b9bd9091b5a4c3e80d (127.10.25.2:35235)
I20260806 19:35:35.732614 17906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27ca366ed45b4abc9617ec06055006e7" candidate_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e787046186fe47b9bd9091b5a4c3e80d"
I20260806 19:35:35.732614 18054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27ca366ed45b4abc9617ec06055006e7" candidate_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90efef159644406821b0eab58afcf43"
I20260806 19:35:35.732872 18054 raft_consensus.cc:3037] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:35.732872 17906 raft_consensus.cc:3037] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:35.736829 18054 raft_consensus.cc:2454] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3dbd5f87907f42d9b9799cd6cdb1bf0c in term 1.
I20260806 19:35:35.736829 17906 raft_consensus.cc:2454] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3dbd5f87907f42d9b9799cd6cdb1bf0c in term 1.
I20260806 19:35:35.737780 17688 leader_election.cc:302] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dbd5f87907f42d9b9799cd6cdb1bf0c, e787046186fe47b9bd9091b5a4c3e80d; no voters:
I20260806 19:35:35.738544 18211 raft_consensus.cc:2781] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:35.739357 18211 raft_consensus.cc:686] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 1 LEADER]: Becoming Leader. State: Replica: 3dbd5f87907f42d9b9799cd6cdb1bf0c, State: Running, Role: LEADER
I20260806 19:35:35.740303 18211 consensus_queue.cc:237] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } }
I20260806 19:35:35.741729 18213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.740853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.1:38711)
0806 19:35:35.741178 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.741412 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196}
I20260806 19:35:35.745764 17618 catalog_manager.cc:5495] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3dbd5f87907f42d9b9799cd6cdb1bf0c (127.10.25.1). New cstate: current_term: 1 leader_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3dbd5f87907f42d9b9799cd6cdb1bf0c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 35055 } health_report { overall_health: HEALTHY } } }
I20260806 19:35:35.786284 18200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.778740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49800 (local address 127.10.25.1:35055)
0806 19:35:35.778996 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:35.779001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:35.779078 (+ 77us) server_negotiation.cc:407] Connection header received
0806 19:35:35.779208 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:35.779212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:35.779301 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:35.779466 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:35:35.779946 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.780692 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.782456 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.782948 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.783072 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:35.784132 (+ 1060us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:35:35.785224 (+ 1092us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:35:35.785556 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:35.785758 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 19:35:35.785934 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 19:35:35.792006 18054 raft_consensus.cc:1261] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43 [term 1 FOLLOWER]: Refusing update from remote peer 3dbd5f87907f42d9b9799cd6cdb1bf0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:35:35.792392 17906 raft_consensus.cc:1261] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d [term 1 FOLLOWER]: Refusing update from remote peer 3dbd5f87907f42d9b9799cd6cdb1bf0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:35:35.792936 18214 consensus_queue.cc:1035] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90efef159644406821b0eab58afcf43" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 45683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:35.793438 18211 consensus_queue.cc:1035] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e787046186fe47b9bd9091b5a4c3e80d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:35.797246 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.796097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.3:33585)
0806 19:35:35.796791 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.796942 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":563}
I20260806 19:35:35.798987 18204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.798349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.2:35225)
0806 19:35:35.798579 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.798763 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 19:35:35.809857 18218 mvcc.cc:205] Tried to move back new op lower bound from 7315640056995127296 to 7315640056800104448. Current Snapshot: MvccSnapshot[applied={T|T < 7315640056995127296}]
I20260806 19:35:35.811851 18209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.811270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.3:51471)
0806 19:35:35.811526 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.811634 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260806 19:35:35.812283 18219 mvcc.cc:205] Tried to move back new op lower bound from 7315640056995127296 to 7315640056800104448. Current Snapshot: MvccSnapshot[applied={T|T < 7315640056995127296}]
I20260806 19:35:35.814283 18204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.813706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.2:38049)
0806 19:35:35.813995 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.814096 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260806 19:35:35.828354 18213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.827194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.1:49271)
0806 19:35:35.828027 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.828137 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":717}
I20260806 19:35:35.830363 18213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.829858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.10.25.1:35269)
0806 19:35:35.830091 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.830184 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260806 19:35:35.842053 17617 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:35:35.842617 17617 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:35:35.851954 17617 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c4294fa83ab48fda6e92f76562d93b5: Sending DeleteTablet for 3 replicas of tablet 27ca366ed45b4abc9617ec06055006e7
I20260806 19:35:35.853631 17885 tablet_service.cc:1514] Processing DeleteTablet for tablet 27ca366ed45b4abc9617ec06055006e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:35 UTC) from {username='oryx', principal='oryx/127.10.25.60@KRBTEST.COM'} at 127.0.0.1:56474
I20260806 19:35:35.853641 18031 tablet_service.cc:1514] Processing DeleteTablet for tablet 27ca366ed45b4abc9617ec06055006e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:35 UTC) from {username='oryx', principal='oryx/127.10.25.60@KRBTEST.COM'} at 127.0.0.1:43506
I20260806 19:35:35.854576 17736 tablet_service.cc:1514] Processing DeleteTablet for tablet 27ca366ed45b4abc9617ec06055006e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:35 UTC) from {username='oryx', principal='oryx/127.10.25.60@KRBTEST.COM'} at 127.0.0.1:49796
I20260806 19:35:35.855077 18227 tablet_replica.cc:330] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d: stopping tablet replica
I20260806 19:35:35.855435 18227 raft_consensus.cc:2229] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:35.855981 18227 raft_consensus.cc:2258] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:35.858124 18227 ts_tablet_manager.cc:1905] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:35.861176 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 17597
I20260806 19:35:35.867336 18229 tablet_replica.cc:330] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: stopping tablet replica
I20260806 19:35:35.867640 18229 raft_consensus.cc:2229] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:35:35.871789 18229 raft_consensus.cc:2258] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:35.874221 18227 ts_tablet_manager.cc:1918] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:35.874485 18227 log.cc:1196] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/27ca366ed45b4abc9617ec06055006e7
I20260806 19:35:35.874475 18228 tablet_replica.cc:330] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43: stopping tablet replica
I20260806 19:35:35.874588 18229 ts_tablet_manager.cc:1905] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:35.874729 18228 raft_consensus.cc:2229] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:35.875475 18228 raft_consensus.cc:2258] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:35.875908 18227 ts_tablet_manager.cc:1939] T 27ca366ed45b4abc9617ec06055006e7 P e787046186fe47b9bd9091b5a4c3e80d: Deleting consensus metadata
I20260806 19:35:35.877147 18228 ts_tablet_manager.cc:1905] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:35.877862 17601 catalog_manager.cc:4841] TS e787046186fe47b9bd9091b5a4c3e80d (127.10.25.2:35235): tablet 27ca366ed45b4abc9617ec06055006e7 (table test-table [id=a2b5a69f3ebd4dae8cbe85bf31b59676]) successfully deleted
I20260806 19:35:35.885983 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.885288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:59756)
0806 19:35:35.885693 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.885801 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260806 19:35:35.885962 18228 ts_tablet_manager.cc:1918] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:35.886123 18228 log.cc:1196] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/27ca366ed45b4abc9617ec06055006e7
I20260806 19:35:35.886591 18228 ts_tablet_manager.cc:1939] T 27ca366ed45b4abc9617ec06055006e7 P e90efef159644406821b0eab58afcf43: Deleting consensus metadata
I20260806 19:35:35.888163 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.887438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:59764)
0806 19:35:35.887788 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:35.887905 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202}
I20260806 19:35:35.888603 17602 catalog_manager.cc:4841] TS e90efef159644406821b0eab58afcf43 (127.10.25.3:45683): tablet 27ca366ed45b4abc9617ec06055006e7 (table test-table [id=a2b5a69f3ebd4dae8cbe85bf31b59676]) successfully deleted
I20260806 19:35:35.894310 18229 ts_tablet_manager.cc:1918] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:35.894594 18229 log.cc:1196] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/27ca366ed45b4abc9617ec06055006e7
I20260806 19:35:35.895309 18229 ts_tablet_manager.cc:1939] T 27ca366ed45b4abc9617ec06055006e7 P 3dbd5f87907f42d9b9799cd6cdb1bf0c: Deleting consensus metadata
I20260806 19:35:35.897121 17603 catalog_manager.cc:4841] TS 3dbd5f87907f42d9b9799cd6cdb1bf0c (127.10.25.1:35055): tablet 27ca366ed45b4abc9617ec06055006e7 (table test-table [id=a2b5a69f3ebd4dae8cbe85bf31b59676]) successfully deleted
I20260806 19:35:35.899081 18200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:35.863605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49808 (local address 127.10.25.1:35055)
0806 19:35:35.864058 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:35.864067 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:35.868702 (+ 4635us) server_negotiation.cc:407] Connection header received
0806 19:35:35.869084 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:35.869090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:35.869223 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:35.869298 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:35.872157 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.873322 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.879230 (+ 5908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:35.880012 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:35.880201 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:35.885834 (+ 5633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:35.885855 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:35.885877 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:35.885941 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:35.892033 (+ 6092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:35.895175 (+ 3142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:35.895193 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:35.895199 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:35.895623 (+ 424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:35.896299 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:35.896304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:35.896308 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:35.896945 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:35.897799 (+ 854us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:35.898201 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 19:35:35.898569 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 19:35:35.900218 17798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:36.022689 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 17597
I20260806 19:35:36.046679 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 17821
I20260806 19:35:36.063859 18198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:36.047559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56490 (local address 127.10.25.2:35235)
0806 19:35:36.047751 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:36.047756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:36.047776 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:35:36.047933 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:36.047937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:36.048004 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:36.048097 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:36.049619 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:36.050416 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:36.052174 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:36.052612 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:36.052737 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:36.056369 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:36.056391 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:36.056409 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:36.056457 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:36.059948 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:36.061180 (+ 1232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:36.061190 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:36.061192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:36.061359 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:36.062013 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:36.062018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:36.062020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:36.062591 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:36.062956 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:36.063393 (+ 437us) server_negotiation.cc:299] Negotiation successful
0806 19:35:36.063615 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 19:35:36.064622 17948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:36.197471 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 17821
I20260806 19:35:36.228408 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 17967
I20260806 19:35:36.247526 18234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:36.229750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43510 (local address 127.10.25.3:45683)
0806 19:35:36.230268 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:36.230273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:36.230292 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:35:36.230354 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:36.230357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:36.230420 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:36.230585 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:36.232106 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:36.233049 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:36.235236 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:36.235892 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:36.236101 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:36.239427 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:36.239449 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:36.239466 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:36.239529 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:36.244110 (+ 4581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:36.245148 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:36.245156 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:36.245159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:36.245319 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:36.245810 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:36.245814 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:36.245816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:36.246223 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:36.246578 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:36.246914 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 19:35:36.247174 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":284,"threads_started":1}
I20260806 19:35:36.248404 18095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:36.351384 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 17967
I20260806 19:35:36.374464 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 17282
I20260806 19:35:36.376041 17455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:36.472944 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:36.472115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:59774)
0806 19:35:36.472668 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:36.472778 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":309,"threads_started":1}
I20260806 19:35:36.474563 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:36.474201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39721 (local address 127.0.0.1:59776)
0806 19:35:36.474346 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:36.474421 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 19:35:36.501873 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 17282
I20260806 19:35:36.528148 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 17388
I20260806 19:35:36.529659 17648 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:36.655762 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 17388
2026-08-06T19:35:36Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11634 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:35:36 dist-test-slave-ss4n krb5kdc[18243](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:35:36 dist-test-slave-ss4n krb5kdc[18243](info): set up 2 sockets
Aug 06 19:35:36 dist-test-slave-ss4n krb5kdc[18243](info): commencing operation
krb5kdc: starting...
W20260806 19:35:37.901115 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:35:38 dist-test-slave-ss4n krb5kdc[18243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:35:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:35:38Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.62.keytab.
Entry for principal oryx/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.62.keytab.
I20260806 19:35:38.203553 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:38305
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.62.keytab
--principal=oryx/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:42419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:38305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:38.565145 18256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:38.565546 18256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:38.565732 18256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:38.573369 18256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:35:38.573570 18256 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 19:35:38.573665 18256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:38.573772 18256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:38.573869 18256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:35:38.573968 18256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:35:38.584563 18256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:38305
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.62.keytab
--principal=oryx/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:38305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:38.586038 18256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:38.587517 18256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:38.602928 18261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:38.602877 18264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:38.602831 18262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:38.603972 18256 server_base.cc:1034] running on GCE node
I20260806 19:35:38.605010 18256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:38.606851 18256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:38.608193 18256 hybrid_clock.cc:648] HybridClock initialized: now 1786044938608099 us; error 69 us; skew 500 ppm
I20260806 19:35:38.608656 18256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:38 dist-test-slave-ss4n krb5kdc[18243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044938, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:38.612545 18256 init.cc:377] Logged in from keytab as oryx/127.10.25.62@KRBTEST.COM (short username oryx)
I20260806 19:35:38.614657 18256 webserver.cc:460] Webserver started at http://127.10.25.62:34857/ using document root <none> and password file <none>
I20260806 19:35:38.615521 18256 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:38.615628 18256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:38.616032 18256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:38.619580 18256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "cfc9e7847d204227805df682e7d74e0c"
format_stamp: "Formatted at 2026-08-06 19:35:38 on dist-test-slave-ss4n"
server_key: "4a75eafb44df39bf3de4a1ff5d8649bb"
server_key_iv: "de703099b0d623ed73b4342b137e8385"
server_key_version: "encryptionkey@0"
I20260806 19:35:38.620611 18256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "cfc9e7847d204227805df682e7d74e0c"
format_stamp: "Formatted at 2026-08-06 19:35:38 on dist-test-slave-ss4n"
server_key: "4a75eafb44df39bf3de4a1ff5d8649bb"
server_key_iv: "de703099b0d623ed73b4342b137e8385"
server_key_version: "encryptionkey@0"
I20260806 19:35:38.627074 18256 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 19:35:38.631245 18272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:38.633039 18256 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:35:38.633306 18256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "cfc9e7847d204227805df682e7d74e0c"
format_stamp: "Formatted at 2026-08-06 19:35:38 on dist-test-slave-ss4n"
server_key: "4a75eafb44df39bf3de4a1ff5d8649bb"
server_key_iv: "de703099b0d623ed73b4342b137e8385"
server_key_version: "encryptionkey@0"
I20260806 19:35:38.633615 18256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:38.650072 18256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:38.655776 18256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:38.656149 18256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:38.685240 18256 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:38305
I20260806 19:35:38.685348 18334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:38305 every 8 connection(s)
I20260806 19:35:38.686965 18256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:35:38.692054 18335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:38.695688 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 18256
I20260806 19:35:38.695940 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:35:38.696663 10340 external_mini_cluster.cc:1428] Setting key 605fc0d16ef5139517ce8bd577ac6391
I20260806 19:35:38.707180 18335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c: Bootstrap starting.
I20260806 19:35:38.712688 18335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:38.714948 18335 log.cc:824] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c: Log is configured to *not* fsync() on all Append() calls
Aug 06 19:35:38 dist-test-slave-ss4n krb5kdc[18243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM
I20260806 19:35:38.718850 18335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c: No bootstrap required, opened a new log
I20260806 19:35:38.727133 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:38.699111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59324 (local address 127.10.25.62:38305)
0806 19:35:38.699997 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:38.700010 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:38.700057 (+ 47us) server_negotiation.cc:407] Connection header received
0806 19:35:38.702028 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:38.702056 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:38.702553 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:38.702956 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:38.705195 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:38.707242 (+ 2047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:38.709721 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:38.710605 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:38.710786 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:38.717566 (+ 6780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:38.717590 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:38.717621 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:38.717674 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:38.721752 (+ 4078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:38.722908 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:38.722917 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:38.722925 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:38.723189 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:38.723695 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:38.723700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:38.723702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:38.724444 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:38.724719 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:38.725288 (+ 569us) server_negotiation.cc:299] Negotiation successful
0806 19:35:38.725902 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":409,"threads_started":1}
I20260806 19:35:38.728593 18335 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc9e7847d204227805df682e7d74e0c" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38305 } }
I20260806 19:35:38.728981 18335 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:38.729143 18335 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfc9e7847d204227805df682e7d74e0c, State: Initialized, Role: FOLLOWER
I20260806 19:35:38.729863 18335 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc9e7847d204227805df682e7d74e0c" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38305 } }
I20260806 19:35:38.730147 18335 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:35:38.730314 18335 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:35:38.730530 18335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:38.733306 18335 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc9e7847d204227805df682e7d74e0c" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38305 } }
I20260806 19:35:38.733948 18335 leader_election.cc:302] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfc9e7847d204227805df682e7d74e0c; no voters:
I20260806 19:35:38.734623 18335 leader_election.cc:288] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:35:38.734817 18340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:38.735564 18340 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [term 1 LEADER]: Becoming Leader. State: Replica: cfc9e7847d204227805df682e7d74e0c, State: Running, Role: LEADER
I20260806 19:35:38.736186 18340 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc9e7847d204227805df682e7d74e0c" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38305 } }
I20260806 19:35:38.736970 18335 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:35:38.740355 18341 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfc9e7847d204227805df682e7d74e0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc9e7847d204227805df682e7d74e0c" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38305 } } }
I20260806 19:35:38.740470 18342 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfc9e7847d204227805df682e7d74e0c. Latest consensus state: current_term: 1 leader_uuid: "cfc9e7847d204227805df682e7d74e0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfc9e7847d204227805df682e7d74e0c" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38305 } } }
I20260806 19:35:38.740859 18341 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:38.740856 18342 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:38.741896 18348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:35:38.749552 18348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:35:38.762676 18348 catalog_manager.cc:1347] Generated new cluster ID: 9bd581c28269427ba32b157a1746aee3
I20260806 19:35:38.762843 18348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:35:38.793429 18348 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:35:38.795301 18348 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:35:38.808306 18348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c: Generated new TSK 0
I20260806 19:35:38.809362 18348 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.1.keytab.
Entry for principal oryx/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.1.keytab.
I20260806 19:35:38.891543 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:38305
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.1.keytab
--principal=oryx/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:42419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:39.294029 18361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:39.294361 18361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:39.294552 18361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:39.302369 18361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:39.302507 18361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:39.302690 18361 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:35:39.302786 18361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:35:39.314080 18361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.1.keytab
--principal=oryx/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.25.62:38305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:39.315582 18361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:39.317096 18361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:39.332891 18366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:39.332891 18367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:39.333099 18369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:39.334458 18361 server_base.cc:1034] running on GCE node
I20260806 19:35:39.335187 18361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:39.336256 18361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:39.337647 18361 hybrid_clock.cc:648] HybridClock initialized: now 1786044939337549 us; error 57 us; skew 500 ppm
I20260806 19:35:39.338050 18361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:39 dist-test-slave-ss4n krb5kdc[18243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044939, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:39.342152 18361 init.cc:377] Logged in from keytab as oryx/127.10.25.1@KRBTEST.COM (short username oryx)
I20260806 19:35:39.344485 18361 webserver.cc:460] Webserver started at http://127.10.25.1:38517/ using document root <none> and password file <none>
I20260806 19:35:39.345947 18361 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:39.346095 18361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:39.346477 18361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:39.350073 18361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "245c506ffdef4b2a9f98946fdbc1b0c8"
format_stamp: "Formatted at 2026-08-06 19:35:39 on dist-test-slave-ss4n"
server_key: "844a77b9f82cfa21b8d02fcabf23993e"
server_key_iv: "5fdafac1ed81d5a184f3f6c3a0e54901"
server_key_version: "encryptionkey@0"
I20260806 19:35:39.351080 18361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "245c506ffdef4b2a9f98946fdbc1b0c8"
format_stamp: "Formatted at 2026-08-06 19:35:39 on dist-test-slave-ss4n"
server_key: "844a77b9f82cfa21b8d02fcabf23993e"
server_key_iv: "5fdafac1ed81d5a184f3f6c3a0e54901"
server_key_version: "encryptionkey@0"
I20260806 19:35:39.357282 18361 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:35:39.361727 18376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:39.364216 18361 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:35:39.364567 18361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "245c506ffdef4b2a9f98946fdbc1b0c8"
format_stamp: "Formatted at 2026-08-06 19:35:39 on dist-test-slave-ss4n"
server_key: "844a77b9f82cfa21b8d02fcabf23993e"
server_key_iv: "5fdafac1ed81d5a184f3f6c3a0e54901"
server_key_version: "encryptionkey@0"
I20260806 19:35:39.364936 18361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:39.392446 18361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:39.398231 18361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:39.398581 18361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:39.401898 18361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:39.402060 18361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:39.402266 18361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:39.402355 18361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:35:39 dist-test-slave-ss4n krb5kdc[18243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044939, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM
I20260806 19:35:39.434573 18385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:39.402946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:50731 (local address 127.10.25.62:38305)
0806 19:35:39.403524 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:39.403529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:39.405184 (+ 1655us) server_negotiation.cc:407] Connection header received
0806 19:35:39.408487 (+ 3303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:39.408492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:39.408735 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:39.408886 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:39.412354 (+ 3468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:39.413575 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:39.417907 (+ 4332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:39.418518 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:39.418715 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:39.426156 (+ 7441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:39.426163 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:39.426167 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:39.426213 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:39.429330 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:39.430904 (+ 1574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:39.430909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:39.430912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:39.431083 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:39.431846 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:39.431853 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:39.431857 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:39.432284 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:39.432470 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:39.433971 (+ 1501us) server_negotiation.cc:299] Negotiation successful
0806 19:35:39.434190 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":309,"threads_started":1}
I20260806 19:35:39.437793 18386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:39.403767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:38305 (local address 127.10.25.1:50731)
0806 19:35:39.404925 (+ 1158us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:39.404988 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:39.407897 (+ 2909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:39.409311 (+ 1414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:39.409328 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:39.410629 (+ 1301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:39.412031 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:39.412076 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:39.413866 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:39.413876 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:39.417601 (+ 3725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:39.417643 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:39.418933 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:39.418946 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:39.419418 (+ 472us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:39.420494 (+ 1076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:39.420562 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:39.425687 (+ 5125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:39.429608 (+ 3921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:39.429626 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:39.429658 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:39.430680 (+ 1022us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:39.431265 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:39.431270 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:39.431274 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:39.431658 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:39.432474 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:39.432486 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:39.433264 (+ 778us) client_negotiation.cc:769] Sending connection context
0806 19:35:39.433813 (+ 549us) client_negotiation.cc:240] Negotiation successful
0806 19:35:39.434845 (+ 1032us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":473,"threads_started":1}
I20260806 19:35:39.469360 18385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:39.447643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:54391 (local address 127.10.25.62:38305)
0806 19:35:39.448031 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:39.448042 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:39.448837 (+ 795us) server_negotiation.cc:407] Connection header received
0806 19:35:39.449159 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:39.449166 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:39.449271 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:39.449416 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:39.451150 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:39.452127 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:39.455614 (+ 3487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:39.456540 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:39.456703 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:39.461293 (+ 4590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:39.461300 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:39.461304 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:39.461346 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:39.465166 (+ 3820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:39.466996 (+ 1830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:39.467003 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:39.467007 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:39.467176 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:39.467621 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:39.467627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:39.467631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:39.467987 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:39.468179 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:39.468679 (+ 500us) server_negotiation.cc:299] Negotiation successful
0806 19:35:39.468953 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207}
I20260806 19:35:39.469564 18453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:39.447668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:38305 (local address 127.10.25.1:54391)
0806 19:35:39.448649 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:39.448681 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:39.448906 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:39.449472 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:39.449509 (+ 37us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:39.450362 (+ 853us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:39.450904 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:39.450923 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:39.452382 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:39.452390 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:39.455279 (+ 2889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:39.455313 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:39.457137 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:39.457151 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:39.457409 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:39.458780 (+ 1371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:39.458818 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:39.460866 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:39.465664 (+ 4798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:39.465674 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:39.465680 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:39.466822 (+ 1142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:39.467318 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:39.467325 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:39.467329 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:39.467481 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:39.468129 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:39.468134 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:39.468362 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 19:35:39.468525 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 19:35:39.468945 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":504,"threads_started":1}
I20260806 19:35:39.478873 18361 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:32855
I20260806 19:35:39.478920 18499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:32855 every 8 connection(s)
I20260806 19:35:39.480355 18361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:35:39.482971 18500 heartbeater.cc:347] Connected to a master server at 127.10.25.62:38305
I20260806 19:35:39.483278 18500 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:39.484091 18500 heartbeater.cc:511] Master 127.10.25.62:38305 requested a full tablet report, sending...
I20260806 19:35:39.486459 18289 ts_manager.cc:194] Registered new tserver with Master: 245c506ffdef4b2a9f98946fdbc1b0c8 (127.10.25.1:32855)
I20260806 19:35:39.488029 18289 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:50731
I20260806 19:35:39.489041 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 18361
I20260806 19:35:39.489275 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:35:39.489996 10340 external_mini_cluster.cc:1428] Setting key ae605d93d206d00b92fa05e09509b314
WARNING: no policy specified for oryx/127.10.25.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.25.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.2.keytab.
Entry for principal oryx/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.2.keytab.
I20260806 19:35:39.555536 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:38305
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.2.keytab
--principal=oryx/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:42419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:39.962880 18504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:39.963301 18504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:39.963500 18504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:39.972779 18504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:39.972990 18504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:39.973157 18504 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:35:39.973315 18504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:35:39.986337 18504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.2.keytab
--principal=oryx/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.25.62:38305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:39.988216 18504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:39.990469 18504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:40.005461 18513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:40.005759 18510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:40.005672 18511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:40.006985 18504 server_base.cc:1034] running on GCE node
I20260806 19:35:40.007797 18504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:40.009307 18504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:40.010646 18504 hybrid_clock.cc:648] HybridClock initialized: now 1786044940010555 us; error 64 us; skew 500 ppm
I20260806 19:35:40.011018 18504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:40 dist-test-slave-ss4n krb5kdc[18243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044940, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:40.015008 18504 init.cc:377] Logged in from keytab as oryx/127.10.25.2@KRBTEST.COM (short username oryx)
I20260806 19:35:40.016912 18504 webserver.cc:460] Webserver started at http://127.10.25.2:45617/ using document root <none> and password file <none>
I20260806 19:35:40.017879 18504 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:40.018046 18504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:40.018512 18504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:40.022070 18504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "7e2b53951c714d069800b3d40fdc5dbc"
format_stamp: "Formatted at 2026-08-06 19:35:40 on dist-test-slave-ss4n"
server_key: "ea3a59bd54e365ce37a29eeda3e529a7"
server_key_iv: "c78a8c55a7e2366959b0e25758084faf"
server_key_version: "encryptionkey@0"
I20260806 19:35:40.023101 18504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "7e2b53951c714d069800b3d40fdc5dbc"
format_stamp: "Formatted at 2026-08-06 19:35:40 on dist-test-slave-ss4n"
server_key: "ea3a59bd54e365ce37a29eeda3e529a7"
server_key_iv: "c78a8c55a7e2366959b0e25758084faf"
server_key_version: "encryptionkey@0"
I20260806 19:35:40.029475 18504 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 19:35:40.034821 18520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:40.036653 18504 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:35:40.036952 18504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "7e2b53951c714d069800b3d40fdc5dbc"
format_stamp: "Formatted at 2026-08-06 19:35:40 on dist-test-slave-ss4n"
server_key: "ea3a59bd54e365ce37a29eeda3e529a7"
server_key_iv: "c78a8c55a7e2366959b0e25758084faf"
server_key_version: "encryptionkey@0"
I20260806 19:35:40.037298 18504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:40.071666 18504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:40.077114 18504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:40.077404 18504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:40.080878 18504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:40.081043 18504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:40.081274 18504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:40.081441 18504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:35:40 dist-test-slave-ss4n krb5kdc[18243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044940, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM
I20260806 19:35:40.115020 18529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:40.082357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:43501 (local address 127.10.25.62:38305)
0806 19:35:40.083098 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:40.083105 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:40.084934 (+ 1829us) server_negotiation.cc:407] Connection header received
0806 19:35:40.088990 (+ 4056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:40.089002 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:40.089151 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:40.089349 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:40.093606 (+ 4257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.094915 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.098465 (+ 3550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.099237 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.099426 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:40.105763 (+ 6337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:40.105770 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:40.105773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:40.105814 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:40.110101 (+ 4287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:40.111336 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:40.111343 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:40.111348 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:40.111569 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:40.112413 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:40.112419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:40.112423 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:40.112812 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:40.113014 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:40.114242 (+ 1228us) server_negotiation.cc:299] Negotiation successful
0806 19:35:40.114557 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":407,"threads_started":1}
I20260806 19:35:40.116981 18530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:40.083398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:38305 (local address 127.10.25.2:43501)
0806 19:35:40.084618 (+ 1220us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:40.084736 (+ 118us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:40.088358 (+ 3622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:40.090070 (+ 1712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:40.090090 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:40.091887 (+ 1797us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:40.093232 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:40.093280 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.095129 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.095134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:40.098129 (+ 2995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:40.098151 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.099859 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.099872 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:40.100288 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:40.101635 (+ 1347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:40.101681 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:40.105381 (+ 3700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:40.110338 (+ 4957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:40.110354 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:40.110404 (+ 50us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:40.111144 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:40.111762 (+ 618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:40.111775 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:40.111782 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:40.112188 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:40.112979 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:40.112990 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:40.113701 (+ 711us) client_negotiation.cc:769] Sending connection context
0806 19:35:40.114107 (+ 406us) client_negotiation.cc:240] Negotiation successful
0806 19:35:40.114861 (+ 754us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"thread_start_us":553,"threads_started":1}
I20260806 19:35:40.133466 18504 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:44195
I20260806 19:35:40.133646 18642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:44195 every 8 connection(s)
I20260806 19:35:40.135250 18504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:35:40.139092 18643 heartbeater.cc:347] Connected to a master server at 127.10.25.62:38305
I20260806 19:35:40.139437 18643 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:40.139787 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 18504
I20260806 19:35:40.139943 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:35:40.140383 10340 external_mini_cluster.cc:1428] Setting key c01073977ec94fe41d88b4c789cf038d
I20260806 19:35:40.140444 18643 heartbeater.cc:511] Master 127.10.25.62:38305 requested a full tablet report, sending...
I20260806 19:35:40.143395 18289 ts_manager.cc:194] Registered new tserver with Master: 7e2b53951c714d069800b3d40fdc5dbc (127.10.25.2:44195)
I20260806 19:35:40.144841 18289 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:43501
I20260806 19:35:40.147774 18529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:40.126984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:48675 (local address 127.10.25.62:38305)
0806 19:35:40.127576 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:40.127584 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:40.128888 (+ 1304us) server_negotiation.cc:407] Connection header received
0806 19:35:40.129241 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:40.129247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:40.129335 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:40.129532 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:40.131171 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.131987 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.135214 (+ 3227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.135812 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.135986 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:40.139781 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:40.139791 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:40.139799 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:40.139859 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:40.144177 (+ 4318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:40.145177 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:40.145185 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:40.145188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:40.145393 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:40.146022 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:40.146028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:40.146032 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:40.146421 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:40.146613 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:40.147103 (+ 490us) server_negotiation.cc:299] Negotiation successful
0806 19:35:40.147376 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383}
I20260806 19:35:40.148186 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:40.127017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:38305 (local address 127.10.25.2:48675)
0806 19:35:40.128612 (+ 1595us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:40.128667 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:40.128953 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:40.129579 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:40.129585 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:40.130562 (+ 977us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:40.130942 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:40.130954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.132281 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.132290 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:40.134890 (+ 2600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:40.134924 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.136089 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.136104 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:40.136336 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:40.137349 (+ 1013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:40.137380 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:40.139433 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:40.144368 (+ 4935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:40.144377 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:40.144381 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:40.144876 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:40.145600 (+ 724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:40.145605 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:40.145609 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:40.145806 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:40.146573 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:40.146579 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:40.146812 (+ 233us) client_negotiation.cc:769] Sending connection context
0806 19:35:40.146976 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 19:35:40.147376 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1295,"thread_start_us":1077,"threads_started":1}
WARNING: no policy specified for oryx/127.10.25.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.25.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.3.keytab.
Entry for principal oryx/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.3.keytab.
I20260806 19:35:40.214500 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:38305
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.3.keytab
--principal=oryx/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:42419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:35:40.491662 18500 heartbeater.cc:503] Master 127.10.25.62:38305 was elected leader, sending a full tablet report...
W20260806 19:35:40.611436 18647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:40.611810 18647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:40.611928 18647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:40.621162 18647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:40.621307 18647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:40.621395 18647 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 19:35:40.621532 18647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:35:40.632144 18647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.3.keytab
--principal=oryx/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.25.62:38305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:40.633591 18647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:40.635069 18647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:40.651783 18653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:40.651784 18656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:40.651832 18654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:40.653968 18647 server_base.cc:1034] running on GCE node
I20260806 19:35:40.654765 18647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:40.656001 18647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:40.657332 18647 hybrid_clock.cc:648] HybridClock initialized: now 1786044940657239 us; error 75 us; skew 500 ppm
I20260806 19:35:40.657927 18647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:40 dist-test-slave-ss4n krb5kdc[18243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044940, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:40.667413 18647 init.cc:377] Logged in from keytab as oryx/127.10.25.3@KRBTEST.COM (short username oryx)
I20260806 19:35:40.669924 18647 webserver.cc:460] Webserver started at http://127.10.25.3:37775/ using document root <none> and password file <none>
I20260806 19:35:40.670774 18647 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:40.670957 18647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:40.671347 18647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:40.674903 18647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "aa98c491ecf648d3961345af0d38c26d"
format_stamp: "Formatted at 2026-08-06 19:35:40 on dist-test-slave-ss4n"
server_key: "4a1c7313b554ef6f14100696e85e382f"
server_key_iv: "41ed827881cefc355b6ea80a41784804"
server_key_version: "encryptionkey@0"
I20260806 19:35:40.675894 18647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "aa98c491ecf648d3961345af0d38c26d"
format_stamp: "Formatted at 2026-08-06 19:35:40 on dist-test-slave-ss4n"
server_key: "4a1c7313b554ef6f14100696e85e382f"
server_key_iv: "41ed827881cefc355b6ea80a41784804"
server_key_version: "encryptionkey@0"
I20260806 19:35:40.682705 18647 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 19:35:40.687323 18663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:40.689126 18647 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 19:35:40.689352 18647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "aa98c491ecf648d3961345af0d38c26d"
format_stamp: "Formatted at 2026-08-06 19:35:40 on dist-test-slave-ss4n"
server_key: "4a1c7313b554ef6f14100696e85e382f"
server_key_iv: "41ed827881cefc355b6ea80a41784804"
server_key_version: "encryptionkey@0"
I20260806 19:35:40.689730 18647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:40.717751 18647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:40.723385 18647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:40.723784 18647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:40.728665 18647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:40.728823 18647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:40.729024 18647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:40.729115 18647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:35:40 dist-test-slave-ss4n krb5kdc[18243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044940, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM
I20260806 19:35:40.767249 18673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:40.731032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:51655 (local address 127.10.25.62:38305)
0806 19:35:40.738370 (+ 7338us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:40.738388 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:40.738434 (+ 46us) server_negotiation.cc:407] Connection header received
0806 19:35:40.738542 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:40.738547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:40.738646 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:40.738840 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:40.741943 (+ 3103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.743665 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.748828 (+ 5163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.750048 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.750245 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:40.756922 (+ 6677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:40.756933 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:40.756941 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:40.757005 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:40.761429 (+ 4424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:40.763267 (+ 1838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:40.763277 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:40.763286 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:40.763555 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:40.764353 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:40.764361 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:40.764366 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:40.764815 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:40.764988 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:40.766497 (+ 1509us) server_negotiation.cc:299] Negotiation successful
0806 19:35:40.766749 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7003,"thread_start_us":6701,"threads_started":1}
I20260806 19:35:40.769374 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:40.730234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:38305 (local address 127.10.25.3:51655)
0806 19:35:40.731436 (+ 1202us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:40.731513 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:40.736217 (+ 4704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:40.739369 (+ 3152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:40.739385 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:40.740431 (+ 1046us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:40.741585 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:40.741669 (+ 84us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.744048 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.744056 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:40.748421 (+ 4365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:40.748466 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.750348 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.750354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:40.750803 (+ 449us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:40.751869 (+ 1066us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:40.751907 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:40.756355 (+ 4448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:40.761919 (+ 5564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:40.761936 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:40.761967 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:40.762978 (+ 1011us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:40.763764 (+ 786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:40.763772 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:40.763782 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:40.764155 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:40.765152 (+ 997us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:40.765159 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:40.765868 (+ 709us) client_negotiation.cc:769] Sending connection context
0806 19:35:40.767630 (+ 1762us) client_negotiation.cc:240] Negotiation successful
0806 19:35:40.768152 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"thread_start_us":530,"threads_started":1}
I20260806 19:35:40.798592 18673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:40.780825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:45023 (local address 127.10.25.62:38305)
0806 19:35:40.781216 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:40.781228 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:40.782359 (+ 1131us) server_negotiation.cc:407] Connection header received
0806 19:35:40.782775 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:40.782786 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:40.782935 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:40.783139 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:40.784432 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.786250 (+ 1818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.787843 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.788333 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.788493 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:40.791413 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:40.791425 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:40.791434 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:40.791489 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:40.795658 (+ 4169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:40.796435 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:40.796444 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:40.796447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:40.796618 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:40.797014 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:40.797020 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:40.797023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:40.797325 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:40.797486 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:40.797955 (+ 469us) server_negotiation.cc:299] Negotiation successful
0806 19:35:40.798208 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260806 19:35:40.798833 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:40.780666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:38305 (local address 127.10.25.3:45023)
0806 19:35:40.782185 (+ 1519us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:40.782228 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:40.782489 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:40.783229 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:40.783236 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:40.783816 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:40.784174 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:40.784185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.786493 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.786498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:40.787636 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:40.787651 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.788566 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.788573 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:40.788790 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:40.789639 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:40.789672 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:40.791090 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:40.795837 (+ 4747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:40.795843 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:40.795846 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:40.796295 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:40.796743 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:40.796750 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:40.796758 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:40.796893 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:40.797461 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:40.797465 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:40.797690 (+ 225us) client_negotiation.cc:769] Sending connection context
0806 19:35:40.797847 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 19:35:40.798314 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1293,"thread_start_us":1030,"threads_started":1}
I20260806 19:35:40.800843 18647 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:34493
I20260806 19:35:40.800853 18786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:34493 every 8 connection(s)
I20260806 19:35:40.802328 18647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:35:40.805289 18787 heartbeater.cc:347] Connected to a master server at 127.10.25.62:38305
I20260806 19:35:40.805601 18787 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:40.806316 18787 heartbeater.cc:511] Master 127.10.25.62:38305 requested a full tablet report, sending...
I20260806 19:35:40.807962 18289 ts_manager.cc:194] Registered new tserver with Master: aa98c491ecf648d3961345af0d38c26d (127.10.25.3:34493)
I20260806 19:35:40.808509 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 18647
I20260806 19:35:40.808904 18289 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:51655
I20260806 19:35:40.808965 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:35:40.809659 10340 external_mini_cluster.cc:1428] Setting key 603659399f7ec5453e3a2cbcc2741205
I20260806 19:35:40.813750 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 19:35:40 dist-test-slave-ss4n krb5kdc[18243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044940, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 19:35:40 dist-test-slave-ss4n krb5kdc[18243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:40 dist-test-slave-ss4n krb5kdc[18243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 19:35:40.886504 18673 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:40.872796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59326 (local address 127.10.25.62:38305)
0806 19:35:40.873258 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:40.873264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:40.873312 (+ 48us) server_negotiation.cc:407] Connection header received
0806 19:35:40.873390 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:40.873395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:40.873648 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:40.873825 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:40.875632 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.876460 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.878165 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.878656 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.878844 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:40.886104 (+ 7260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59326: BlockingRecv error: recv got EOF from 127.0.0.1:59326 (error 108)
Metrics: {"server-negotiator.queue_time_us":159}
Aug 06 19:35:40 dist-test-slave-ss4n krb5kdc[18243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044940, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM
I20260806 19:35:40.910902 18673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:40.891908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59340 (local address 127.10.25.62:38305)
0806 19:35:40.892192 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:40.892199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:40.892437 (+ 238us) server_negotiation.cc:407] Connection header received
0806 19:35:40.892589 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:40.892594 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:40.892667 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:40.892766 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:40.894564 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.895733 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.897125 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.897782 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.897908 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:40.902929 (+ 5021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:40.902935 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:40.902939 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:40.902989 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:40.906836 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:40.908198 (+ 1362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:40.908204 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:40.908207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:40.908456 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:40.909099 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:40.909104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:40.909106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:40.909458 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:40.909839 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:40.910364 (+ 525us) server_negotiation.cc:299] Negotiation successful
0806 19:35:40.910635 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260806 19:35:40.916849 18289 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:35:40.924604 18289 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:35:40.958237 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:40.942609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:32855 (local address 127.0.0.1:41694)
0806 19:35:40.943641 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:40.943698 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:40.943893 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:40.945726 (+ 1833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:40.945742 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:40.945793 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:40.946487 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:40.946508 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.949464 (+ 2956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.949475 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:40.953349 (+ 3874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:40.953383 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.956568 (+ 3185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.956580 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:40.956958 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:40.957056 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 19:35:40.957239 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 19:35:40.957598 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":389,"threads_started":1}
I20260806 19:35:40.960963 18808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:40.942948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41694 (local address 127.10.25.1:32855)
0806 19:35:40.944551 (+ 1603us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:40.944561 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:40.944593 (+ 32us) server_negotiation.cc:407] Connection header received
0806 19:35:40.944712 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:40.944723 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:40.945368 (+ 645us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:40.946777 (+ 1409us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:40.947076 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.949204 (+ 2128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.953815 (+ 4611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.956200 (+ 2385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.959888 (+ 3688us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:40.960264 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:40.960365 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 19:35:40.960610 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1412,"thread_start_us":1270,"threads_started":1}
I20260806 19:35:40.966071 18429 tablet_service.cc:1467] Processing CreateTablet for tablet 69be87aab0d544609d217ab6b26b9892 (DEFAULT_TABLE table=test-table [id=49cb78737a324a64bb870cd8492f8e3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:40.967995 18429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69be87aab0d544609d217ab6b26b9892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:40.969951 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:40.943538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58936 (local address 127.10.25.2:44195)
0806 19:35:40.960871 (+ 17333us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:40.960881 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:40.960916 (+ 35us) server_negotiation.cc:407] Connection header received
0806 19:35:40.961040 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:40.961052 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:40.961416 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:40.961717 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:40.962711 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.964481 (+ 1770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.967077 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.968626 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.968821 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:40.969070 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:40.969338 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 19:35:40.969572 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17118,"thread_start_us":18762,"threads_started":1}
I20260806 19:35:40.970070 18811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:40.942975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:44195 (local address 127.0.0.1:58936)
0806 19:35:40.954458 (+ 11483us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:40.954489 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:40.954661 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:40.961714 (+ 7053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:40.961721 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:40.961752 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:40.962283 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:40.962303 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.964663 (+ 2360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.964669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:40.966765 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:40.966815 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.968872 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.968879 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:40.969091 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:40.969137 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 19:35:40.969281 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 19:35:40.969453 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11238,"mutex_wait_us":1,"thread_start_us":19483,"threads_started":1}
I20260806 19:35:40.971271 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:40.944581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:34493 (local address 127.0.0.1:49346)
0806 19:35:40.959027 (+ 14446us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:40.959059 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:40.959245 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:40.963018 (+ 3773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:40.963022 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:40.963054 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:40.963305 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:40.963317 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.965032 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.965038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:40.967532 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:40.967550 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.969870 (+ 2320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.969877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:40.970529 (+ 652us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:40.970585 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 19:35:40.970702 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 19:35:40.970868 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14274,"mutex_wait_us":81,"thread_start_us":9363,"threads_started":1}
I20260806 19:35:40.973377 18812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:40.945270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49346 (local address 127.10.25.3:34493)
0806 19:35:40.962314 (+ 17044us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:40.962325 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:40.962369 (+ 44us) server_negotiation.cc:407] Connection header received
0806 19:35:40.962495 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:40.962503 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:40.962789 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:40.963024 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:40.963485 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.964770 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.967781 (+ 3011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:40.969644 (+ 1863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:40.971920 (+ 2276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:40.972234 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:40.972322 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 19:35:40.972876 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16798,"thread_start_us":14785,"threads_started":1}
I20260806 19:35:40.976651 18572 tablet_service.cc:1467] Processing CreateTablet for tablet 69be87aab0d544609d217ab6b26b9892 (DEFAULT_TABLE table=test-table [id=49cb78737a324a64bb870cd8492f8e3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:40.978354 18715 tablet_service.cc:1467] Processing CreateTablet for tablet 69be87aab0d544609d217ab6b26b9892 (DEFAULT_TABLE table=test-table [id=49cb78737a324a64bb870cd8492f8e3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:40.978953 18572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69be87aab0d544609d217ab6b26b9892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:40.979804 18715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69be87aab0d544609d217ab6b26b9892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:40.988227 18813 tablet_bootstrap.cc:492] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8: Bootstrap starting.
I20260806 19:35:40.993008 18813 tablet_bootstrap.cc:654] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:40.995077 18813 log.cc:824] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:40.995437 18814 tablet_bootstrap.cc:492] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d: Bootstrap starting.
I20260806 19:35:40.997043 18815 tablet_bootstrap.cc:492] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc: Bootstrap starting.
I20260806 19:35:41.000022 18814 tablet_bootstrap.cc:654] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:41.001067 18815 tablet_bootstrap.cc:654] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:41.001688 18814 log.cc:824] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:41.001811 18813 tablet_bootstrap.cc:492] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8: No bootstrap required, opened a new log
I20260806 19:35:41.002187 18813 ts_tablet_manager.cc:1397] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260806 19:35:41.002614 18815 log.cc:824] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:41.006652 18815 tablet_bootstrap.cc:492] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc: No bootstrap required, opened a new log
I20260806 19:35:41.007019 18815 ts_tablet_manager.cc:1397] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260806 19:35:41.010160 18814 tablet_bootstrap.cc:492] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d: No bootstrap required, opened a new log
I20260806 19:35:41.010577 18814 ts_tablet_manager.cc:1397] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.009s
I20260806 19:35:41.015623 18813 raft_consensus.cc:348] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.016240 18813 raft_consensus.cc:374] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:41.016356 18813 raft_consensus.cc:729] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c506ffdef4b2a9f98946fdbc1b0c8, State: Initialized, Role: FOLLOWER
I20260806 19:35:41.017194 18813 consensus_queue.cc:260] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.019546 18813 ts_tablet_manager.cc:1428] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8: Time spent starting tablet: real 0.017s user 0.011s sys 0.004s
I20260806 19:35:41.019721 18815 raft_consensus.cc:348] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.020385 18815 raft_consensus.cc:374] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:41.020613 18815 raft_consensus.cc:729] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e2b53951c714d069800b3d40fdc5dbc, State: Initialized, Role: FOLLOWER
I20260806 19:35:41.021173 18814 raft_consensus.cc:348] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.021389 18815 consensus_queue.cc:260] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.021708 18814 raft_consensus.cc:374] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:41.021847 18814 raft_consensus.cc:729] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa98c491ecf648d3961345af0d38c26d, State: Initialized, Role: FOLLOWER
I20260806 19:35:41.022589 18814 consensus_queue.cc:260] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.023950 18787 heartbeater.cc:503] Master 127.10.25.62:38305 was elected leader, sending a full tablet report...
I20260806 19:35:41.024874 18814 ts_tablet_manager.cc:1428] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260806 19:35:41.027220 18643 heartbeater.cc:503] Master 127.10.25.62:38305 was elected leader, sending a full tablet report...
I20260806 19:35:41.028087 18815 ts_tablet_manager.cc:1428] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc: Time spent starting tablet: real 0.021s user 0.016s sys 0.004s
W20260806 19:35:41.055280 18788 tablet.cc:2369] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:35:41.139626 18644 tablet.cc:2369] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:35:41.233897 18501 tablet.cc:2369] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:35:41.334085 18821 raft_consensus.cc:482] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:35:41.334430 18821 raft_consensus.cc:504] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.336542 18821 leader_election.cc:288] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 245c506ffdef4b2a9f98946fdbc1b0c8 (127.10.25.1:32855), 7e2b53951c714d069800b3d40fdc5dbc (127.10.25.2:44195)
I20260806 19:35:41.345118 18808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:41.336903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:48237 (local address 127.10.25.1:32855)
0806 19:35:41.337211 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:41.337221 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:41.337758 (+ 537us) server_negotiation.cc:407] Connection header received
0806 19:35:41.338089 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:41.338094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:41.338217 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:41.338315 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:41.339037 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:41.339856 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:41.342882 (+ 3026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:41.343864 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:41.344026 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:41.344127 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:41.344648 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 19:35:41.344809 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 19:35:41.345186 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:41.337519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:45219 (local address 127.10.25.2:44195)
0806 19:35:41.337811 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:41.337818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:41.338281 (+ 463us) server_negotiation.cc:407] Connection header received
0806 19:35:41.338439 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:41.338444 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:41.338534 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:41.338654 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:41.339464 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:41.340408 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:41.342907 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:41.344085 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:41.344254 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:41.344392 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:41.344743 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 19:35:41.344901 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 19:35:41.345312 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:41.336546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:32855 (local address 127.10.25.3:48237)
0806 19:35:41.337625 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:41.337653 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:41.337891 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:41.338400 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:41.338405 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:41.338438 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:41.338790 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:41.338809 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:41.340255 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:41.340261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:41.342607 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:41.342636 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:41.344105 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:41.344114 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:41.344409 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:41.344461 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 19:35:41.344567 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 19:35:41.344750 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"mutex_wait_us":143,"thread_start_us":1145,"threads_started":1}
I20260806 19:35:41.345636 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:41.336852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:44195 (local address 127.10.25.3:45219)
0806 19:35:41.338095 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:41.338112 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:41.338274 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:41.338669 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:41.338672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:41.338693 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:41.339110 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:41.339124 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:41.340620 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:41.340624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:41.342672 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:41.342687 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:41.344350 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:41.344355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:41.344505 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:41.344532 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 19:35:41.344679 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 19:35:41.344829 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1080,"mutex_wait_us":57,"thread_start_us":500,"threads_started":1}
I20260806 19:35:41.346505 18454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69be87aab0d544609d217ab6b26b9892" candidate_uuid: "aa98c491ecf648d3961345af0d38c26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" is_pre_election: true
I20260806 19:35:41.346746 18592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69be87aab0d544609d217ab6b26b9892" candidate_uuid: "aa98c491ecf648d3961345af0d38c26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2b53951c714d069800b3d40fdc5dbc" is_pre_election: true
I20260806 19:35:41.347364 18454 raft_consensus.cc:2454] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa98c491ecf648d3961345af0d38c26d in term 0.
I20260806 19:35:41.347518 18592 raft_consensus.cc:2454] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa98c491ecf648d3961345af0d38c26d in term 0.
I20260806 19:35:41.348346 18665 leader_election.cc:302] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c506ffdef4b2a9f98946fdbc1b0c8, aa98c491ecf648d3961345af0d38c26d; no voters:
I20260806 19:35:41.348846 18821 raft_consensus.cc:2781] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:35:41.349030 18821 raft_consensus.cc:482] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:35:41.349162 18821 raft_consensus.cc:3037] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:41.351718 18821 raft_consensus.cc:504] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.352468 18821 leader_election.cc:288] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [CANDIDATE]: Term 1 election: Requested vote from peers 245c506ffdef4b2a9f98946fdbc1b0c8 (127.10.25.1:32855), 7e2b53951c714d069800b3d40fdc5dbc (127.10.25.2:44195)
I20260806 19:35:41.353076 18454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69be87aab0d544609d217ab6b26b9892" candidate_uuid: "aa98c491ecf648d3961345af0d38c26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8"
I20260806 19:35:41.353152 18592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69be87aab0d544609d217ab6b26b9892" candidate_uuid: "aa98c491ecf648d3961345af0d38c26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2b53951c714d069800b3d40fdc5dbc"
I20260806 19:35:41.353328 18454 raft_consensus.cc:3037] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:41.353443 18592 raft_consensus.cc:3037] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:41.356371 18592 raft_consensus.cc:2454] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa98c491ecf648d3961345af0d38c26d in term 1.
I20260806 19:35:41.357028 18664 leader_election.cc:302] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e2b53951c714d069800b3d40fdc5dbc, aa98c491ecf648d3961345af0d38c26d; no voters:
I20260806 19:35:41.357131 18454 raft_consensus.cc:2454] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa98c491ecf648d3961345af0d38c26d in term 1.
I20260806 19:35:41.357424 18821 raft_consensus.cc:2781] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:41.358006 18821 raft_consensus.cc:686] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [term 1 LEADER]: Becoming Leader. State: Replica: aa98c491ecf648d3961345af0d38c26d, State: Running, Role: LEADER
I20260806 19:35:41.358566 18821 consensus_queue.cc:237] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.364500 18287 catalog_manager.cc:5495] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d reported cstate change: term changed from 0 to 1, leader changed from <none> to aa98c491ecf648d3961345af0d38c26d (127.10.25.3). New cstate: current_term: 1 leader_uuid: "aa98c491ecf648d3961345af0d38c26d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } health_report { overall_health: HEALTHY } } }
I20260806 19:35:41.423125 18812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:41.416166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49348 (local address 127.10.25.3:34493)
0806 19:35:41.416403 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:41.416408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:41.416743 (+ 335us) server_negotiation.cc:407] Connection header received
0806 19:35:41.416925 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:41.416929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:41.417036 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:41.417205 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:35:41.417740 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:41.418530 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:41.420220 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:41.420796 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:41.420974 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:41.421599 (+ 625us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:35:41.422344 (+ 745us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:35:41.422525 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:41.422692 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 19:35:41.422794 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 19:35:41.434482 18454 raft_consensus.cc:1261] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 FOLLOWER]: Refusing update from remote peer aa98c491ecf648d3961345af0d38c26d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:35:41.434484 18592 raft_consensus.cc:1261] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc [term 1 FOLLOWER]: Refusing update from remote peer aa98c491ecf648d3961345af0d38c26d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:35:41.435712 18824 consensus_queue.cc:1035] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:41.436156 18821 consensus_queue.cc:1035] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:41.455286 18828 mvcc.cc:205] Tried to move back new op lower bound from 7315640080103989248 to 7315640079809257472. Current Snapshot: MvccSnapshot[applied={T|T < 7315640080103989248}]
I20260806 19:35:41.460076 18830 mvcc.cc:205] Tried to move back new op lower bound from 7315640080103989248 to 7315640079809257472. Current Snapshot: MvccSnapshot[applied={T|T < 7315640080103989248}]
I20260806 19:35:41.475219 18289 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:35:41.475564 18289 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:35:41.480166 18289 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c: Sending DeleteTablet for 3 replicas of tablet 69be87aab0d544609d217ab6b26b9892
I20260806 19:35:41.481794 18572 tablet_service.cc:1514] Processing DeleteTablet for tablet 69be87aab0d544609d217ab6b26b9892 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:41 UTC) from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:58936
I20260806 19:35:41.482542 18429 tablet_service.cc:1514] Processing DeleteTablet for tablet 69be87aab0d544609d217ab6b26b9892 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:41 UTC) from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:41694
I20260806 19:35:41.483117 18715 tablet_service.cc:1514] Processing DeleteTablet for tablet 69be87aab0d544609d217ab6b26b9892 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:41 UTC) from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:49346
I20260806 19:35:41.484293 18289 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:35:41.485039 18289 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:35:41.496127 18714 tablet_service.cc:1467] Processing CreateTablet for tablet 8f846a243d8b4ee897c2ef4edf8c6f80 (DEFAULT_TABLE table=test-table [id=b8b6f4185b3f478587b047bcbab62762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:41.496169 18428 tablet_service.cc:1467] Processing CreateTablet for tablet 8f846a243d8b4ee897c2ef4edf8c6f80 (DEFAULT_TABLE table=test-table [id=b8b6f4185b3f478587b047bcbab62762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:41.496788 18714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f846a243d8b4ee897c2ef4edf8c6f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:41.496966 18428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f846a243d8b4ee897c2ef4edf8c6f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:41.499439 18838 tablet_replica.cc:330] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc: stopping tablet replica
I20260806 19:35:41.500138 18839 tablet_replica.cc:330] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8: stopping tablet replica
I20260806 19:35:41.501004 18839 raft_consensus.cc:2229] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:41.502249 18572 tablet_service.cc:1467] Processing CreateTablet for tablet 8f846a243d8b4ee897c2ef4edf8c6f80 (DEFAULT_TABLE table=test-table [id=b8b6f4185b3f478587b047bcbab62762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:41.502872 18572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f846a243d8b4ee897c2ef4edf8c6f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:41.505352 18814 tablet_bootstrap.cc:492] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d: Bootstrap starting.
I20260806 19:35:41.507733 18814 tablet_bootstrap.cc:654] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:41.501681 18838 raft_consensus.cc:2229] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:41.508517 18838 raft_consensus.cc:2258] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:41.505777 18813 tablet_bootstrap.cc:492] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8: Bootstrap starting.
I20260806 19:35:41.511891 18813 tablet_bootstrap.cc:654] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:41.515360 18839 raft_consensus.cc:2258] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:41.518324 18840 tablet_replica.cc:330] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d: stopping tablet replica
I20260806 19:35:41.518893 18840 raft_consensus.cc:2229] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:35:41.519783 18840 raft_consensus.cc:2258] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:41.521914 18840 ts_tablet_manager.cc:1905] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:41.522665 18839 ts_tablet_manager.cc:1905] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:41.523335 18838 ts_tablet_manager.cc:1905] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:41.526469 18815 tablet_bootstrap.cc:492] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc: Bootstrap starting.
I20260806 19:35:41.531978 18815 tablet_bootstrap.cc:654] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:41.539108 18813 tablet_bootstrap.cc:492] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8: No bootstrap required, opened a new log
I20260806 19:35:41.539443 18813 ts_tablet_manager.cc:1397] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.008s
I20260806 19:35:41.540701 18813 raft_consensus.cc:348] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.541093 18813 raft_consensus.cc:374] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:41.541277 18813 raft_consensus.cc:729] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c506ffdef4b2a9f98946fdbc1b0c8, State: Initialized, Role: FOLLOWER
I20260806 19:35:41.541591 18840 ts_tablet_manager.cc:1918] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:41.541875 18840 log.cc:1196] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/69be87aab0d544609d217ab6b26b9892
I20260806 19:35:41.542569 18840 ts_tablet_manager.cc:1939] T 69be87aab0d544609d217ab6b26b9892 P aa98c491ecf648d3961345af0d38c26d: Deleting consensus metadata
I20260806 19:35:41.542531 18813 consensus_queue.cc:260] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.544840 18813 ts_tablet_manager.cc:1428] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260806 19:35:41.544787 18274 catalog_manager.cc:4841] TS aa98c491ecf648d3961345af0d38c26d (127.10.25.3:34493): tablet 69be87aab0d544609d217ab6b26b9892 (table test-table [id=49cb78737a324a64bb870cd8492f8e3f]) successfully deleted
I20260806 19:35:41.546573 18815 tablet_bootstrap.cc:492] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc: No bootstrap required, opened a new log
I20260806 19:35:41.546859 18815 ts_tablet_manager.cc:1397] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.004s
I20260806 19:35:41.547943 18815 raft_consensus.cc:348] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.548309 18815 raft_consensus.cc:374] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:41.548470 18815 raft_consensus.cc:729] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e2b53951c714d069800b3d40fdc5dbc, State: Initialized, Role: FOLLOWER
I20260806 19:35:41.548921 18815 consensus_queue.cc:260] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.549582 18814 tablet_bootstrap.cc:492] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d: No bootstrap required, opened a new log
I20260806 19:35:41.549786 18814 ts_tablet_manager.cc:1397] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d: Time spent bootstrapping tablet: real 0.045s user 0.010s sys 0.028s
I20260806 19:35:41.550854 18814 raft_consensus.cc:348] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.551123 18814 raft_consensus.cc:374] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:41.551247 18814 raft_consensus.cc:729] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa98c491ecf648d3961345af0d38c26d, State: Initialized, Role: FOLLOWER
I20260806 19:35:41.551636 18839 ts_tablet_manager.cc:1918] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:41.551831 18839 log.cc:1196] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/69be87aab0d544609d217ab6b26b9892
I20260806 19:35:41.551644 18814 consensus_queue.cc:260] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.552500 18839 ts_tablet_manager.cc:1939] T 69be87aab0d544609d217ab6b26b9892 P 245c506ffdef4b2a9f98946fdbc1b0c8: Deleting consensus metadata
I20260806 19:35:41.553006 18815 ts_tablet_manager.cc:1428] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260806 19:35:41.553644 18814 ts_tablet_manager.cc:1428] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260806 19:35:41.557723 18838 ts_tablet_manager.cc:1918] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:41.557936 18838 log.cc:1196] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/69be87aab0d544609d217ab6b26b9892
I20260806 19:35:41.558606 18838 ts_tablet_manager.cc:1939] T 69be87aab0d544609d217ab6b26b9892 P 7e2b53951c714d069800b3d40fdc5dbc: Deleting consensus metadata
I20260806 19:35:41.559542 18274 catalog_manager.cc:4841] TS 245c506ffdef4b2a9f98946fdbc1b0c8 (127.10.25.1:32855): tablet 69be87aab0d544609d217ab6b26b9892 (table test-table [id=49cb78737a324a64bb870cd8492f8e3f]) successfully deleted
I20260806 19:35:41.560434 18273 catalog_manager.cc:4841] TS 7e2b53951c714d069800b3d40fdc5dbc (127.10.25.2:44195): tablet 69be87aab0d544609d217ab6b26b9892 (table test-table [id=49cb78737a324a64bb870cd8492f8e3f]) successfully deleted
I20260806 19:35:41.675367 18821 raft_consensus.cc:482] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:35:41.675721 18821 raft_consensus.cc:504] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.676476 18821 leader_election.cc:288] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 245c506ffdef4b2a9f98946fdbc1b0c8 (127.10.25.1:32855), 7e2b53951c714d069800b3d40fdc5dbc (127.10.25.2:44195)
I20260806 19:35:41.677084 18592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f846a243d8b4ee897c2ef4edf8c6f80" candidate_uuid: "aa98c491ecf648d3961345af0d38c26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2b53951c714d069800b3d40fdc5dbc" is_pre_election: true
I20260806 19:35:41.677121 18454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f846a243d8b4ee897c2ef4edf8c6f80" candidate_uuid: "aa98c491ecf648d3961345af0d38c26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" is_pre_election: true
I20260806 19:35:41.677343 18592 raft_consensus.cc:2454] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa98c491ecf648d3961345af0d38c26d in term 0.
I20260806 19:35:41.677433 18454 raft_consensus.cc:2454] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa98c491ecf648d3961345af0d38c26d in term 0.
I20260806 19:35:41.677951 18665 leader_election.cc:302] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 245c506ffdef4b2a9f98946fdbc1b0c8, 7e2b53951c714d069800b3d40fdc5dbc, aa98c491ecf648d3961345af0d38c26d; no voters:
I20260806 19:35:41.678400 18821 raft_consensus.cc:2781] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:35:41.678555 18821 raft_consensus.cc:482] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:35:41.678690 18821 raft_consensus.cc:3037] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:41.681044 18821 raft_consensus.cc:504] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.681804 18821 leader_election.cc:288] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [CANDIDATE]: Term 1 election: Requested vote from peers 245c506ffdef4b2a9f98946fdbc1b0c8 (127.10.25.1:32855), 7e2b53951c714d069800b3d40fdc5dbc (127.10.25.2:44195)
I20260806 19:35:41.682432 18592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f846a243d8b4ee897c2ef4edf8c6f80" candidate_uuid: "aa98c491ecf648d3961345af0d38c26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2b53951c714d069800b3d40fdc5dbc"
I20260806 19:35:41.682439 18454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f846a243d8b4ee897c2ef4edf8c6f80" candidate_uuid: "aa98c491ecf648d3961345af0d38c26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8"
I20260806 19:35:41.682667 18592 raft_consensus.cc:3037] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:41.682684 18454 raft_consensus.cc:3037] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:41.685333 18592 raft_consensus.cc:2454] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa98c491ecf648d3961345af0d38c26d in term 1.
I20260806 19:35:41.686978 18454 raft_consensus.cc:2454] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa98c491ecf648d3961345af0d38c26d in term 1.
I20260806 19:35:41.687394 18664 leader_election.cc:302] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e2b53951c714d069800b3d40fdc5dbc, aa98c491ecf648d3961345af0d38c26d; no voters:
I20260806 19:35:41.687915 18821 raft_consensus.cc:2781] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:41.688140 18821 raft_consensus.cc:686] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [term 1 LEADER]: Becoming Leader. State: Replica: aa98c491ecf648d3961345af0d38c26d, State: Running, Role: LEADER
I20260806 19:35:41.688475 18821 consensus_queue.cc:237] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.692525 18289 catalog_manager.cc:5495] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d reported cstate change: term changed from 0 to 1, leader changed from <none> to aa98c491ecf648d3961345af0d38c26d (127.10.25.3). New cstate: current_term: 1 leader_uuid: "aa98c491ecf648d3961345af0d38c26d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } health_report { overall_health: HEALTHY } } }
Aug 06 19:35:41 dist-test-slave-ss4n krb5kdc[18243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044938, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.62@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM
I20260806 19:35:41.761413 18851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:41.741717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:38305 (local address 127.0.0.1:59346)
0806 19:35:41.743218 (+ 1501us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:41.743249 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:41.743482 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:41.743997 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:41.744001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:41.744807 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:41.745066 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:41.745077 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:41.746864 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:41.746868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:41.748092 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:41.748107 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:41.749356 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:41.749362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:41.749527 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:41.750340 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:41.750364 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:41.753122 (+ 2758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:41.758140 (+ 5018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:41.758156 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:41.758163 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:41.758643 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:41.759255 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:41.759262 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:41.759266 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:41.759532 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:41.760387 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:41.760396 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:41.760593 (+ 197us) client_negotiation.cc:769] Sending connection context
0806 19:35:41.760780 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 19:35:41.761075 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1316,"thread_start_us":1160,"threads_started":1}
I20260806 19:35:41.761816 18850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:41.741671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59346 (local address 127.10.25.62:38305)
0806 19:35:41.742613 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:41.742624 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:41.743401 (+ 777us) server_negotiation.cc:407] Connection header received
0806 19:35:41.743671 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:41.743678 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:41.743767 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:41.743907 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:41.745294 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:41.746679 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:41.748286 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:41.749072 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:41.749243 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:41.753441 (+ 4198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:41.753453 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:41.753459 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:41.753531 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:41.757920 (+ 4389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:41.758799 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:41.758809 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:41.758815 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:41.758980 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:41.759720 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:41.759726 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:41.759731 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:41.760191 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:41.760373 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:41.760910 (+ 537us) server_negotiation.cc:299] Negotiation successful
0806 19:35:41.761312 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":731,"thread_start_us":465,"threads_started":1}
I20260806 19:35:41.768304 18289 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:59346:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 19:35:41.769549 18289 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:35:41.779167 18428 tablet_service.cc:1467] Processing CreateTablet for tablet e8d77cae0781417c85c67addae80ad5a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0dfeef5babfc47b3bba11c6ef6c546a5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:35:41.779639 18572 tablet_service.cc:1467] Processing CreateTablet for tablet e8d77cae0781417c85c67addae80ad5a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0dfeef5babfc47b3bba11c6ef6c546a5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:35:41.779645 18715 tablet_service.cc:1467] Processing CreateTablet for tablet e8d77cae0781417c85c67addae80ad5a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0dfeef5babfc47b3bba11c6ef6c546a5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 19:35:41.779882 18428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8d77cae0781417c85c67addae80ad5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:41.780225 18572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8d77cae0781417c85c67addae80ad5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:41.780290 18715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8d77cae0781417c85c67addae80ad5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:41.789103 18815 tablet_bootstrap.cc:492] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc: Bootstrap starting.
I20260806 19:35:41.789183 18813 tablet_bootstrap.cc:492] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8: Bootstrap starting.
I20260806 19:35:41.789868 18814 tablet_bootstrap.cc:492] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d: Bootstrap starting.
I20260806 19:35:41.791333 18815 tablet_bootstrap.cc:654] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:41.791369 18813 tablet_bootstrap.cc:654] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:41.792066 18814 tablet_bootstrap.cc:654] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:41.801721 18815 tablet_bootstrap.cc:492] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc: No bootstrap required, opened a new log
I20260806 19:35:41.801954 18815 ts_tablet_manager.cc:1397] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260806 19:35:41.802848 18813 tablet_bootstrap.cc:492] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8: No bootstrap required, opened a new log
I20260806 19:35:41.803038 18813 ts_tablet_manager.cc:1397] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.005s
I20260806 19:35:41.802932 18815 raft_consensus.cc:348] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.803216 18815 raft_consensus.cc:374] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:41.803306 18815 raft_consensus.cc:729] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e2b53951c714d069800b3d40fdc5dbc, State: Initialized, Role: FOLLOWER
I20260806 19:35:41.803862 18813 raft_consensus.cc:348] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.804150 18813 raft_consensus.cc:374] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:41.804227 18813 raft_consensus.cc:729] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c506ffdef4b2a9f98946fdbc1b0c8, State: Initialized, Role: FOLLOWER
I20260806 19:35:41.804234 18815 consensus_queue.cc:260] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.804863 18820 tablet_replica.cc:439] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } }
I20260806 19:35:41.805248 18820 tablet_replica.cc:442] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:35:41.806162 18813 consensus_queue.cc:260] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.806886 18819 tablet_replica.cc:439] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } }
I20260806 19:35:41.807379 18819 tablet_replica.cc:442] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:35:41.807912 18813 ts_tablet_manager.cc:1428] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260806 19:35:41.808771 18815 ts_tablet_manager.cc:1428] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260806 19:35:41.831039 18814 tablet_bootstrap.cc:492] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d: No bootstrap required, opened a new log
I20260806 19:35:41.831300 18814 ts_tablet_manager.cc:1397] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d: Time spent bootstrapping tablet: real 0.041s user 0.008s sys 0.005s
I20260806 19:35:41.832489 18814 raft_consensus.cc:348] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.832860 18814 raft_consensus.cc:374] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:41.832975 18814 raft_consensus.cc:729] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa98c491ecf648d3961345af0d38c26d, State: Initialized, Role: FOLLOWER
I20260806 19:35:41.833384 18814 consensus_queue.cc:260] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:41.834033 18821 tablet_replica.cc:439] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } }
I20260806 19:35:41.834461 18821 tablet_replica.cc:442] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:35:41.838606 18814 ts_tablet_manager.cc:1428] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260806 19:35:42.048096 18821 raft_consensus.cc:482] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:35:42.048383 18821 raft_consensus.cc:504] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:42.049329 18821 leader_election.cc:288] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 245c506ffdef4b2a9f98946fdbc1b0c8 (127.10.25.1:32855), 7e2b53951c714d069800b3d40fdc5dbc (127.10.25.2:44195)
I20260806 19:35:42.050203 18592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d77cae0781417c85c67addae80ad5a" candidate_uuid: "aa98c491ecf648d3961345af0d38c26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2b53951c714d069800b3d40fdc5dbc" is_pre_election: true
I20260806 19:35:42.050371 18454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d77cae0781417c85c67addae80ad5a" candidate_uuid: "aa98c491ecf648d3961345af0d38c26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" is_pre_election: true
I20260806 19:35:42.050519 18592 raft_consensus.cc:2454] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa98c491ecf648d3961345af0d38c26d in term 0.
I20260806 19:35:42.050695 18454 raft_consensus.cc:2454] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa98c491ecf648d3961345af0d38c26d in term 0.
I20260806 19:35:42.051193 18664 leader_election.cc:302] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e2b53951c714d069800b3d40fdc5dbc, aa98c491ecf648d3961345af0d38c26d; no voters:
I20260806 19:35:42.051615 18821 raft_consensus.cc:2781] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:35:42.051725 18821 raft_consensus.cc:482] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:35:42.051832 18821 raft_consensus.cc:3037] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:42.055496 18821 raft_consensus.cc:504] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:42.056824 18592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d77cae0781417c85c67addae80ad5a" candidate_uuid: "aa98c491ecf648d3961345af0d38c26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2b53951c714d069800b3d40fdc5dbc"
I20260806 19:35:42.056835 18454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8d77cae0781417c85c67addae80ad5a" candidate_uuid: "aa98c491ecf648d3961345af0d38c26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8"
I20260806 19:35:42.057124 18592 raft_consensus.cc:3037] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:42.057144 18454 raft_consensus.cc:3037] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:42.060894 18592 raft_consensus.cc:2454] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa98c491ecf648d3961345af0d38c26d in term 1.
I20260806 19:35:42.061551 18664 leader_election.cc:302] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e2b53951c714d069800b3d40fdc5dbc, aa98c491ecf648d3961345af0d38c26d; no voters:
I20260806 19:35:42.061740 18454 raft_consensus.cc:2454] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa98c491ecf648d3961345af0d38c26d in term 1.
I20260806 19:35:42.061903 18824 raft_consensus.cc:2781] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:42.062695 18824 raft_consensus.cc:686] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [term 1 LEADER]: Becoming Leader. State: Replica: aa98c491ecf648d3961345af0d38c26d, State: Running, Role: LEADER
I20260806 19:35:42.063028 18824 consensus_queue.cc:237] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:42.063683 18821 leader_election.cc:288] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [CANDIDATE]: Term 1 election: Requested vote from peers 245c506ffdef4b2a9f98946fdbc1b0c8 (127.10.25.1:32855), 7e2b53951c714d069800b3d40fdc5dbc (127.10.25.2:44195)
I20260806 19:35:42.065671 18855 tablet_replica.cc:439] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d: TxnStatusTablet state changed. Reason: New leader aa98c491ecf648d3961345af0d38c26d. Latest consensus state: current_term: 1 leader_uuid: "aa98c491ecf648d3961345af0d38c26d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } }
I20260806 19:35:42.066066 18855 tablet_replica.cc:442] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:35:42.066680 18288 catalog_manager.cc:5495] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d reported cstate change: term changed from 0 to 1, leader changed from <none> to aa98c491ecf648d3961345af0d38c26d (127.10.25.3). New cstate: current_term: 1 leader_uuid: "aa98c491ecf648d3961345af0d38c26d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } health_report { overall_health: HEALTHY } } }
I20260806 19:35:42.071017 18857 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 19:35:42.071415 18857 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 19:35:42.078871 18855 consensus_queue.cc:1035] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:42.096611 18821 consensus_queue.cc:1035] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:42.146486 18851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.128586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:34493 (local address 127.0.0.1:48532)
0806 19:35:42.128903 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:42.128929 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:42.129119 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:42.138714 (+ 9595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:42.138721 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:42.138745 (+ 24us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:35:42.139201 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:42.139221 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.141082 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.141089 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:42.142920 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:42.142942 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.144670 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.144677 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:42.144917 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.145018 (+ 101us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:35:42.145729 (+ 711us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:35:42.145758 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 19:35:42.145824 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 19:35:42.146114 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 19:35:42.147753 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.128553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48532 (local address 127.10.25.3:34493)
0806 19:35:42.137756 (+ 9203us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:42.137762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:42.137823 (+ 61us) server_negotiation.cc:407] Connection header received
0806 19:35:42.137933 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:42.137957 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:42.138059 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:42.138639 (+ 580us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:35:42.139463 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.140870 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.143691 (+ 2821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.144444 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.145254 (+ 810us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.145407 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:35:42.145592 (+ 185us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:35:42.147130 (+ 1538us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:42.147210 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 19:35:42.147303 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9053,"thread_start_us":8863,"threads_started":1}
I20260806 19:35:42.152693 18592 raft_consensus.cc:1261] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc [term 1 FOLLOWER]: Refusing update from remote peer aa98c491ecf648d3961345af0d38c26d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:35:42.152747 18454 raft_consensus.cc:1261] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 FOLLOWER]: Refusing update from remote peer aa98c491ecf648d3961345af0d38c26d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:35:42.153438 18821 consensus_queue.cc:1035] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:42.153872 18824 consensus_queue.cc:1035] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:42.165851 18819 tablet_replica.cc:439] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8: TxnStatusTablet state changed. Reason: New leader aa98c491ecf648d3961345af0d38c26d. Latest consensus state: current_term: 1 leader_uuid: "aa98c491ecf648d3961345af0d38c26d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } }
I20260806 19:35:42.166245 18819 tablet_replica.cc:442] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:35:42.169291 18820 tablet_replica.cc:439] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc: TxnStatusTablet state changed. Reason: New leader aa98c491ecf648d3961345af0d38c26d. Latest consensus state: current_term: 1 leader_uuid: "aa98c491ecf648d3961345af0d38c26d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } }
I20260806 19:35:42.169691 18820 tablet_replica.cc:442] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:35:42.180542 18820 tablet_replica.cc:439] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa98c491ecf648d3961345af0d38c26d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } }
I20260806 19:35:42.181108 18820 tablet_replica.cc:442] T e8d77cae0781417c85c67addae80ad5a P 7e2b53951c714d069800b3d40fdc5dbc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:35:42.183096 18824 tablet_replica.cc:439] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa98c491ecf648d3961345af0d38c26d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } }
I20260806 19:35:42.183482 18824 tablet_replica.cc:442] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:35:42.184587 18821 tablet_replica.cc:439] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa98c491ecf648d3961345af0d38c26d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } }
I20260806 19:35:42.184952 18821 tablet_replica.cc:442] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d: This TxnStatusTablet replica's current role is: LEADER
I20260806 19:35:42.186309 18819 tablet_replica.cc:439] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa98c491ecf648d3961345af0d38c26d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } }
I20260806 19:35:42.186599 18819 tablet_replica.cc:442] T e8d77cae0781417c85c67addae80ad5a P 245c506ffdef4b2a9f98946fdbc1b0c8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 19:35:42.209141 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.201657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:50275 (local address 127.10.25.3:34493)
0806 19:35:42.201920 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:42.201927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:42.202376 (+ 449us) server_negotiation.cc:407] Connection header received
0806 19:35:42.202608 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:42.202612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:42.202682 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:42.202771 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:35:42.203383 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.204379 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.206684 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.207305 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.207425 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.208143 (+ 718us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:35:42.208271 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:35:42.208360 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:42.208661 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 19:35:42.208793 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 19:35:42.209399 18873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.201434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:34493 (local address 127.10.25.3:50275)
0806 19:35:42.202250 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:42.202270 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:42.202448 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:42.202807 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:42.202812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:42.202825 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:35:42.203225 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:42.203240 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.204542 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.204548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:42.206477 (+ 1929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:42.206494 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.207538 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.207545 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:42.207761 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.207850 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:35:42.208390 (+ 540us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:35:42.208402 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 19:35:42.208494 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 19:35:42.208666 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":367,"threads_started":1}
I20260806 19:35:42.286618 18287 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:35:42.286926 18287 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:35:42.291661 18287 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c: Sending DeleteTablet for 3 replicas of tablet 8f846a243d8b4ee897c2ef4edf8c6f80
I20260806 19:35:42.293251 18572 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f846a243d8b4ee897c2ef4edf8c6f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:42 UTC) from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:58936
I20260806 19:35:42.293296 18428 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f846a243d8b4ee897c2ef4edf8c6f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:42 UTC) from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:41694
I20260806 19:35:42.293391 18714 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f846a243d8b4ee897c2ef4edf8c6f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:42 UTC) from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:49346
I20260806 19:35:42.305941 18875 tablet_replica.cc:330] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8: stopping tablet replica
I20260806 19:35:42.306555 18876 tablet_replica.cc:330] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc: stopping tablet replica
I20260806 19:35:42.306944 18876 raft_consensus.cc:2229] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:42.307366 18875 raft_consensus.cc:2229] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:42.307775 18876 raft_consensus.cc:2258] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:42.308145 18875 raft_consensus.cc:2258] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:42.312187 18876 ts_tablet_manager.cc:1905] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:42.315580 18875 ts_tablet_manager.cc:1905] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:42.322037 18876 ts_tablet_manager.cc:1918] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:35:42.322387 18876 log.cc:1196] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/8f846a243d8b4ee897c2ef4edf8c6f80
I20260806 19:35:42.322963 18876 ts_tablet_manager.cc:1939] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 7e2b53951c714d069800b3d40fdc5dbc: Deleting consensus metadata
I20260806 19:35:42.324616 18875 ts_tablet_manager.cc:1918] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:35:42.324790 18875 log.cc:1196] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/8f846a243d8b4ee897c2ef4edf8c6f80
I20260806 19:35:42.324930 18273 catalog_manager.cc:4841] TS 7e2b53951c714d069800b3d40fdc5dbc (127.10.25.2:44195): tablet 8f846a243d8b4ee897c2ef4edf8c6f80 (table test-table [id=b8b6f4185b3f478587b047bcbab62762]) successfully deleted
I20260806 19:35:42.325249 18875 ts_tablet_manager.cc:1939] T 8f846a243d8b4ee897c2ef4edf8c6f80 P 245c506ffdef4b2a9f98946fdbc1b0c8: Deleting consensus metadata
I20260806 19:35:42.326040 18877 tablet_replica.cc:330] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d: stopping tablet replica
I20260806 19:35:42.326378 18877 raft_consensus.cc:2229] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:35:42.326642 18274 catalog_manager.cc:4841] TS 245c506ffdef4b2a9f98946fdbc1b0c8 (127.10.25.1:32855): tablet 8f846a243d8b4ee897c2ef4edf8c6f80 (table test-table [id=b8b6f4185b3f478587b047bcbab62762]) successfully deleted
I20260806 19:35:42.327057 18877 raft_consensus.cc:2258] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:42.328250 18877 ts_tablet_manager.cc:1905] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:42.335224 18877 ts_tablet_manager.cc:1918] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:35:42.335436 18877 log.cc:1196] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/8f846a243d8b4ee897c2ef4edf8c6f80
I20260806 19:35:42.335914 18877 ts_tablet_manager.cc:1939] T 8f846a243d8b4ee897c2ef4edf8c6f80 P aa98c491ecf648d3961345af0d38c26d: Deleting consensus metadata
I20260806 19:35:42.337203 18274 catalog_manager.cc:4841] TS aa98c491ecf648d3961345af0d38c26d (127.10.25.3:34493): tablet 8f846a243d8b4ee897c2ef4edf8c6f80 (table test-table [id=b8b6f4185b3f478587b047bcbab62762]) successfully deleted
Aug 06 19:35:42 dist-test-slave-ss4n krb5kdc[18243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 19:35:42 dist-test-slave-ss4n krb5kdc[18243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:42 dist-test-slave-ss4n krb5kdc[18243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 19:35:42.414886 18884 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:42.387413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53376 (local address 127.10.25.62:38305)
0806 19:35:42.390193 (+ 2780us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:42.390202 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:42.398410 (+ 8208us) server_negotiation.cc:407] Connection header received
0806 19:35:42.398679 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:42.398689 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:42.398816 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:42.398993 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:42.401538 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.402482 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.405060 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.406429 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.406623 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.414466 (+ 7843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53376: BlockingRecv error: recv got EOF from 127.0.0.1:53376 (error 108)
Metrics: {"server-negotiator.queue_time_us":2474,"thread_start_us":2302,"threads_started":1}
Aug 06 19:35:42 dist-test-slave-ss4n krb5kdc[18243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM
I20260806 19:35:42.442464 18884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.419534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.10.25.62:38305)
0806 19:35:42.419757 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:42.419766 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:42.420483 (+ 717us) server_negotiation.cc:407] Connection header received
0806 19:35:42.420760 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:42.420767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:42.420890 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:42.421013 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:42.423301 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.424255 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.426564 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.427059 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.427225 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.432815 (+ 5590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:42.432823 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:42.432831 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:42.432891 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:42.437919 (+ 5028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:42.439069 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:42.439076 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:42.439080 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:42.439314 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:42.440191 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:42.440198 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:42.440202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:42.440648 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:42.440961 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:42.441569 (+ 608us) server_negotiation.cc:299] Negotiation successful
0806 19:35:42.441869 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 19:35:42.447324 18287 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:35:42.448027 18287 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:35:42.458079 18572 tablet_service.cc:1467] Processing CreateTablet for tablet 1027bed24f234b6ab32297414b530e37 (DEFAULT_TABLE table=test-table [id=70e5fb080ea24741a0211e9901f4b4c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:42.458076 18428 tablet_service.cc:1467] Processing CreateTablet for tablet 1027bed24f234b6ab32297414b530e37 (DEFAULT_TABLE table=test-table [id=70e5fb080ea24741a0211e9901f4b4c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:42.458907 18572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1027bed24f234b6ab32297414b530e37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:42.458986 18428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1027bed24f234b6ab32297414b530e37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:42.461694 18714 tablet_service.cc:1467] Processing CreateTablet for tablet 1027bed24f234b6ab32297414b530e37 (DEFAULT_TABLE table=test-table [id=70e5fb080ea24741a0211e9901f4b4c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:42.462648 18714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1027bed24f234b6ab32297414b530e37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:42.473754 18893 tablet_bootstrap.cc:492] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8: Bootstrap starting.
I20260806 19:35:42.474393 18892 tablet_bootstrap.cc:492] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc: Bootstrap starting.
I20260806 19:35:42.476493 18892 tablet_bootstrap.cc:654] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:42.478173 18894 tablet_bootstrap.cc:492] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d: Bootstrap starting.
I20260806 19:35:42.481544 18894 tablet_bootstrap.cc:654] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:42.485374 18892 tablet_bootstrap.cc:492] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc: No bootstrap required, opened a new log
I20260806 19:35:42.485642 18892 ts_tablet_manager.cc:1397] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260806 19:35:42.487227 18892 raft_consensus.cc:348] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:42.487586 18892 raft_consensus.cc:374] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:42.487707 18892 raft_consensus.cc:729] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e2b53951c714d069800b3d40fdc5dbc, State: Initialized, Role: FOLLOWER
I20260806 19:35:42.488035 18892 consensus_queue.cc:260] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:42.488799 18892 ts_tablet_manager.cc:1428] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 19:35:42.492920 18893 tablet_bootstrap.cc:654] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:42.499222 18893 tablet_bootstrap.cc:492] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8: No bootstrap required, opened a new log
I20260806 19:35:42.499457 18893 ts_tablet_manager.cc:1397] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.001s
I20260806 19:35:42.500427 18893 raft_consensus.cc:348] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:42.500749 18893 raft_consensus.cc:374] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:42.500857 18893 raft_consensus.cc:729] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c506ffdef4b2a9f98946fdbc1b0c8, State: Initialized, Role: FOLLOWER
I20260806 19:35:42.501143 18893 consensus_queue.cc:260] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:42.501940 18893 ts_tablet_manager.cc:1428] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 19:35:42.505802 18894 tablet_bootstrap.cc:492] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d: No bootstrap required, opened a new log
I20260806 19:35:42.506074 18894 ts_tablet_manager.cc:1397] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d: Time spent bootstrapping tablet: real 0.028s user 0.011s sys 0.010s
I20260806 19:35:42.506997 18894 raft_consensus.cc:348] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:42.507326 18894 raft_consensus.cc:374] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:42.507469 18894 raft_consensus.cc:729] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa98c491ecf648d3961345af0d38c26d, State: Initialized, Role: FOLLOWER
I20260806 19:35:42.507824 18894 consensus_queue.cc:260] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:42.508720 18894 ts_tablet_manager.cc:1428] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 19:35:42.601187 18819 raft_consensus.cc:482] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:35:42.601562 18819 raft_consensus.cc:504] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:42.604024 18819 leader_election.cc:288] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e2b53951c714d069800b3d40fdc5dbc (127.10.25.2:44195), aa98c491ecf648d3961345af0d38c26d (127.10.25.3:34493)
I20260806 19:35:42.623693 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.605182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:56547 (local address 127.10.25.3:34493)
0806 19:35:42.605415 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:42.605424 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:42.615336 (+ 9912us) server_negotiation.cc:407] Connection header received
0806 19:35:42.615851 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:42.615858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:42.615997 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:42.616119 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:42.616888 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.618367 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.621053 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.622457 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.622611 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.622756 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:42.623170 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 19:35:42.623352 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 19:35:42.624030 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.604793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:37333 (local address 127.10.25.2:44195)
0806 19:35:42.614874 (+ 10081us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:42.614882 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:42.616059 (+ 1177us) server_negotiation.cc:407] Connection header received
0806 19:35:42.616300 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:42.616306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:42.616585 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:42.616728 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:42.617473 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.618417 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.621579 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.622924 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.623085 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.623232 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:42.623518 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 19:35:42.623662 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9888,"thread_start_us":9639,"threads_started":1}
I20260806 19:35:42.624053 18899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.604372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:44195 (local address 127.10.25.1:37333)
0806 19:35:42.615890 (+ 11518us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:42.615912 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:42.616088 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:42.616752 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:42.616756 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:42.616787 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:42.617181 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:42.617191 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.618631 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.618646 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:42.621360 (+ 2714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:42.621377 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.623151 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.623157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:42.623345 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.623369 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 19:35:42.623486 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 19:35:42.623647 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11378,"mutex_wait_us":97,"thread_start_us":10396,"threads_started":1}
I20260806 19:35:42.624991 18898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.604329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:34493 (local address 127.10.25.1:56547)
0806 19:35:42.615156 (+ 10827us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:42.615188 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:42.615499 (+ 311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:42.616254 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:42.616260 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:42.616292 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:42.616655 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:42.616671 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.618552 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.618561 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:42.620845 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:42.620863 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.622704 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.622714 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:42.622918 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.622959 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 19:35:42.624395 (+ 1436us) client_negotiation.cc:240] Negotiation successful
0806 19:35:42.624567 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10624,"mutex_wait_us":110,"thread_start_us":11120,"threads_started":1}
I20260806 19:35:42.624972 18592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1027bed24f234b6ab32297414b530e37" candidate_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2b53951c714d069800b3d40fdc5dbc" is_pre_election: true
I20260806 19:35:42.625270 18592 raft_consensus.cc:2454] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c506ffdef4b2a9f98946fdbc1b0c8 in term 0.
I20260806 19:35:42.625923 18377 leader_election.cc:302] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c506ffdef4b2a9f98946fdbc1b0c8, 7e2b53951c714d069800b3d40fdc5dbc; no voters:
I20260806 19:35:42.626236 18735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1027bed24f234b6ab32297414b530e37" candidate_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa98c491ecf648d3961345af0d38c26d" is_pre_election: true
I20260806 19:35:42.626468 18819 raft_consensus.cc:2781] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:35:42.626610 18819 raft_consensus.cc:482] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:35:42.626729 18819 raft_consensus.cc:3037] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:42.626803 18735 raft_consensus.cc:2454] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c506ffdef4b2a9f98946fdbc1b0c8 in term 0.
I20260806 19:35:42.630605 18819 raft_consensus.cc:504] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:42.631526 18819 leader_election.cc:288] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [CANDIDATE]: Term 1 election: Requested vote from peers 7e2b53951c714d069800b3d40fdc5dbc (127.10.25.2:44195), aa98c491ecf648d3961345af0d38c26d (127.10.25.3:34493)
I20260806 19:35:42.632261 18592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1027bed24f234b6ab32297414b530e37" candidate_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2b53951c714d069800b3d40fdc5dbc"
I20260806 19:35:42.632556 18592 raft_consensus.cc:3037] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:42.632522 18735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1027bed24f234b6ab32297414b530e37" candidate_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa98c491ecf648d3961345af0d38c26d"
I20260806 19:35:42.632772 18735 raft_consensus.cc:3037] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:42.636111 18592 raft_consensus.cc:2454] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c506ffdef4b2a9f98946fdbc1b0c8 in term 1.
I20260806 19:35:42.636649 18735 raft_consensus.cc:2454] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c506ffdef4b2a9f98946fdbc1b0c8 in term 1.
I20260806 19:35:42.636811 18377 leader_election.cc:302] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c506ffdef4b2a9f98946fdbc1b0c8, 7e2b53951c714d069800b3d40fdc5dbc; no voters:
I20260806 19:35:42.637293 18819 raft_consensus.cc:2781] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:42.637869 18819 raft_consensus.cc:686] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 LEADER]: Becoming Leader. State: Replica: 245c506ffdef4b2a9f98946fdbc1b0c8, State: Running, Role: LEADER
I20260806 19:35:42.638438 18819 consensus_queue.cc:237] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } }
I20260806 19:35:42.643249 18287 catalog_manager.cc:5495] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 245c506ffdef4b2a9f98946fdbc1b0c8 (127.10.25.1). New cstate: current_term: 1 leader_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } health_report { overall_health: UNKNOWN } } }
I20260806 19:35:42.689199 18904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.682327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36064 (local address 127.10.25.1:32855)
0806 19:35:42.682992 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:42.682997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:42.683018 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:35:42.683120 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:42.683124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:42.683206 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:42.683412 (+ 206us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:35:42.683919 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.684816 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.686501 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.687007 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.687169 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.687752 (+ 583us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:35:42.688420 (+ 668us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:35:42.688624 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:42.688694 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 19:35:42.688794 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":354,"threads_started":1}
I20260806 19:35:42.693657 18592 raft_consensus.cc:1261] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc [term 1 FOLLOWER]: Refusing update from remote peer 245c506ffdef4b2a9f98946fdbc1b0c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:35:42.693869 18735 raft_consensus.cc:1261] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d [term 1 FOLLOWER]: Refusing update from remote peer 245c506ffdef4b2a9f98946fdbc1b0c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:35:42.694638 18901 consensus_queue.cc:1035] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:42.695191 18819 consensus_queue.cc:1035] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:42.719553 18287 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:35:42.719879 18287 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:35:42.725212 18287 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c: Sending DeleteTablet for 3 replicas of tablet 1027bed24f234b6ab32297414b530e37
I20260806 19:35:42.726245 18714 tablet_service.cc:1514] Processing DeleteTablet for tablet 1027bed24f234b6ab32297414b530e37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:42 UTC) from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:49346
I20260806 19:35:42.726613 18877 tablet_replica.cc:330] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d: stopping tablet replica
I20260806 19:35:42.726836 18877 raft_consensus.cc:2229] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:42.726822 18428 tablet_service.cc:1514] Processing DeleteTablet for tablet 1027bed24f234b6ab32297414b530e37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:42 UTC) from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:41694
I20260806 19:35:42.727097 18877 raft_consensus.cc:2258] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:42.727133 18875 tablet_replica.cc:330] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8: stopping tablet replica
I20260806 19:35:42.727698 18875 raft_consensus.cc:2229] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:35:42.727766 18572 tablet_service.cc:1514] Processing DeleteTablet for tablet 1027bed24f234b6ab32297414b530e37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:42 UTC) from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:58936
I20260806 19:35:42.728089 18876 tablet_replica.cc:330] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc: stopping tablet replica
I20260806 19:35:42.728259 18714 tablet_service.cc:1514] Processing DeleteTablet for tablet 1027bed24f234b6ab32297414b530e37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:42 UTC) from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:49346
I20260806 19:35:42.728296 18876 raft_consensus.cc:2229] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:42.728338 18875 raft_consensus.cc:2258] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:42.728585 18876 raft_consensus.cc:2258] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:42.730082 18876 ts_tablet_manager.cc:1905] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:42.730654 18877 ts_tablet_manager.cc:1905] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:42.730806 18875 ts_tablet_manager.cc:1905] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:42.737607 18287 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:35:42.738557 18287 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:35:42.738991 18875 ts_tablet_manager.cc:1918] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:42.739459 18875 log.cc:1196] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/1027bed24f234b6ab32297414b530e37
I20260806 19:35:42.740155 18875 ts_tablet_manager.cc:1939] T 1027bed24f234b6ab32297414b530e37 P 245c506ffdef4b2a9f98946fdbc1b0c8: Deleting consensus metadata
I20260806 19:35:42.741655 18877 ts_tablet_manager.cc:1918] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:42.741916 18877 log.cc:1196] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/1027bed24f234b6ab32297414b530e37
I20260806 19:35:42.742637 18877 ts_tablet_manager.cc:1939] T 1027bed24f234b6ab32297414b530e37 P aa98c491ecf648d3961345af0d38c26d: Deleting consensus metadata
I20260806 19:35:42.742738 18274 catalog_manager.cc:4841] TS 245c506ffdef4b2a9f98946fdbc1b0c8 (127.10.25.1:32855): tablet 1027bed24f234b6ab32297414b530e37 (table test-table [id=70e5fb080ea24741a0211e9901f4b4c5]) successfully deleted
I20260806 19:35:42.745160 18876 ts_tablet_manager.cc:1918] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:35:42.745834 18876 log.cc:1196] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/1027bed24f234b6ab32297414b530e37
I20260806 19:35:42.746196 18274 catalog_manager.cc:4841] TS aa98c491ecf648d3961345af0d38c26d (127.10.25.3:34493): tablet 1027bed24f234b6ab32297414b530e37 (table test-table [id=70e5fb080ea24741a0211e9901f4b4c5]) successfully deleted
I20260806 19:35:42.746750 18876 ts_tablet_manager.cc:1939] T 1027bed24f234b6ab32297414b530e37 P 7e2b53951c714d069800b3d40fdc5dbc: Deleting consensus metadata
W20260806 19:35:42.747397 18274 catalog_manager.cc:4808] TS aa98c491ecf648d3961345af0d38c26d (127.10.25.3:34493): delete failed for tablet 1027bed24f234b6ab32297414b530e37 because the tablet was not found. No further retry: Not found: Tablet not found: 1027bed24f234b6ab32297414b530e37
I20260806 19:35:42.748323 18273 catalog_manager.cc:4841] TS 7e2b53951c714d069800b3d40fdc5dbc (127.10.25.2:44195): tablet 1027bed24f234b6ab32297414b530e37 (table test-table [id=70e5fb080ea24741a0211e9901f4b4c5]) successfully deleted
I20260806 19:35:42.751189 18714 tablet_service.cc:1467] Processing CreateTablet for tablet b3a1491edbea45658e77beafec99bb2c (DEFAULT_TABLE table=test-table [id=58c8e689c733492bb5a32e9c8f8fa4f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:42.751364 18428 tablet_service.cc:1467] Processing CreateTablet for tablet b3a1491edbea45658e77beafec99bb2c (DEFAULT_TABLE table=test-table [id=58c8e689c733492bb5a32e9c8f8fa4f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:42.751783 18714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3a1491edbea45658e77beafec99bb2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:42.751881 18428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3a1491edbea45658e77beafec99bb2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:42.751871 18572 tablet_service.cc:1467] Processing CreateTablet for tablet b3a1491edbea45658e77beafec99bb2c (DEFAULT_TABLE table=test-table [id=58c8e689c733492bb5a32e9c8f8fa4f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:35:42.752352 18572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3a1491edbea45658e77beafec99bb2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:42.760236 18893 tablet_bootstrap.cc:492] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8: Bootstrap starting.
I20260806 19:35:42.760881 18892 tablet_bootstrap.cc:492] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc: Bootstrap starting.
I20260806 19:35:42.762871 18892 tablet_bootstrap.cc:654] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:42.762862 18893 tablet_bootstrap.cc:654] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:42.763096 18894 tablet_bootstrap.cc:492] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d: Bootstrap starting.
I20260806 19:35:42.765023 18894 tablet_bootstrap.cc:654] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:42.773353 18894 tablet_bootstrap.cc:492] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d: No bootstrap required, opened a new log
I20260806 19:35:42.773770 18894 ts_tablet_manager.cc:1397] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260806 19:35:42.773975 18892 tablet_bootstrap.cc:492] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc: No bootstrap required, opened a new log
I20260806 19:35:42.773983 18893 tablet_bootstrap.cc:492] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8: No bootstrap required, opened a new log
I20260806 19:35:42.774188 18892 ts_tablet_manager.cc:1397] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20260806 19:35:42.774185 18893 ts_tablet_manager.cc:1397] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.007s
I20260806 19:35:42.775122 18892 raft_consensus.cc:348] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } }
I20260806 19:35:42.775117 18893 raft_consensus.cc:348] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } }
I20260806 19:35:42.775435 18893 raft_consensus.cc:374] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:42.775429 18892 raft_consensus.cc:374] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:42.775532 18892 raft_consensus.cc:729] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e2b53951c714d069800b3d40fdc5dbc, State: Initialized, Role: FOLLOWER
I20260806 19:35:42.775535 18893 raft_consensus.cc:729] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c506ffdef4b2a9f98946fdbc1b0c8, State: Initialized, Role: FOLLOWER
I20260806 19:35:42.775919 18893 consensus_queue.cc:260] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } }
I20260806 19:35:42.775846 18892 consensus_queue.cc:260] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } }
I20260806 19:35:42.776816 18892 ts_tablet_manager.cc:1428] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 19:35:42.777151 18894 raft_consensus.cc:348] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } }
I20260806 19:35:42.777562 18894 raft_consensus.cc:374] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:42.777724 18894 raft_consensus.cc:729] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa98c491ecf648d3961345af0d38c26d, State: Initialized, Role: FOLLOWER
I20260806 19:35:42.778127 18894 consensus_queue.cc:260] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } }
I20260806 19:35:42.778570 18893 ts_tablet_manager.cc:1428] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 19:35:42.782127 18894 ts_tablet_manager.cc:1428] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260806 19:35:42.806696 18820 raft_consensus.cc:482] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:35:42.806958 18820 raft_consensus.cc:504] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } }
I20260806 19:35:42.809007 18820 leader_election.cc:288] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa98c491ecf648d3961345af0d38c26d (127.10.25.3:34493), 245c506ffdef4b2a9f98946fdbc1b0c8 (127.10.25.1:32855)
I20260806 19:35:42.823160 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.809447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:42593 (local address 127.10.25.3:34493)
0806 19:35:42.809834 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:42.809846 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:42.811817 (+ 1971us) server_negotiation.cc:407] Connection header received
0806 19:35:42.812144 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:42.812151 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:42.812246 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:42.812311 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:42.813243 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.814508 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.818160 (+ 3652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.819717 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.819868 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.820012 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:42.822618 (+ 2606us) server_negotiation.cc:299] Negotiation successful
0806 19:35:42.822820 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260806 19:35:42.824330 18912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.809262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:34493 (local address 127.10.25.2:42593)
0806 19:35:42.811696 (+ 2434us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:42.811722 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:42.811944 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:42.812554 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:42.812559 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:42.812589 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:42.812997 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:42.813011 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.814853 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.814859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:42.817896 (+ 3037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:42.817921 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.820308 (+ 2387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.820317 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:42.822353 (+ 2036us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.822404 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 19:35:42.823680 (+ 1276us) client_negotiation.cc:240] Negotiation successful
0806 19:35:42.823892 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2278,"thread_start_us":1948,"threads_started":1}
I20260806 19:35:42.825402 18735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3a1491edbea45658e77beafec99bb2c" candidate_uuid: "7e2b53951c714d069800b3d40fdc5dbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa98c491ecf648d3961345af0d38c26d" is_pre_election: true
I20260806 19:35:42.825770 18735 raft_consensus.cc:2454] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e2b53951c714d069800b3d40fdc5dbc in term 0.
I20260806 19:35:42.826458 18522 leader_election.cc:302] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e2b53951c714d069800b3d40fdc5dbc, aa98c491ecf648d3961345af0d38c26d; no voters:
I20260806 19:35:42.826994 18820 raft_consensus.cc:2781] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:35:42.827134 18820 raft_consensus.cc:482] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:35:42.827245 18820 raft_consensus.cc:3037] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:42.831319 18904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.812082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:54749 (local address 127.10.25.1:32855)
0806 19:35:42.812322 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:42.812331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:42.822772 (+ 10441us) server_negotiation.cc:407] Connection header received
0806 19:35:42.823081 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:42.823088 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:42.823168 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:42.823230 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:35:42.824131 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.825298 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.828235 (+ 2937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.829807 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.829952 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.830083 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:42.830778 (+ 695us) server_negotiation.cc:299] Negotiation successful
0806 19:35:42.830968 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 19:35:42.831048 18820 raft_consensus.cc:504] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } }
I20260806 19:35:42.831987 18820 leader_election.cc:288] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [CANDIDATE]: Term 1 election: Requested vote from peers aa98c491ecf648d3961345af0d38c26d (127.10.25.3:34493), 245c506ffdef4b2a9f98946fdbc1b0c8 (127.10.25.1:32855)
I20260806 19:35:42.832202 18913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.812022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:32855 (local address 127.10.25.2:54749)
0806 19:35:42.822650 (+ 10628us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:42.822674 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:42.822890 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:42.823449 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:42.823455 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:42.823481 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:35:42.823917 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:42.823929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.825464 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.825469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:42.827966 (+ 2497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:42.827985 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.830326 (+ 2341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.830334 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:42.830539 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.830576 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 19:35:42.831539 (+ 963us) client_negotiation.cc:240] Negotiation successful
0806 19:35:42.831706 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10471,"mutex_wait_us":4,"thread_start_us":10230,"threads_started":1}
I20260806 19:35:42.833052 18735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3a1491edbea45658e77beafec99bb2c" candidate_uuid: "7e2b53951c714d069800b3d40fdc5dbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa98c491ecf648d3961345af0d38c26d"
I20260806 19:35:42.833348 18735 raft_consensus.cc:3037] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:42.836376 18735 raft_consensus.cc:2454] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e2b53951c714d069800b3d40fdc5dbc in term 1.
I20260806 19:35:42.837765 18522 leader_election.cc:302] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e2b53951c714d069800b3d40fdc5dbc, aa98c491ecf648d3961345af0d38c26d; no voters:
I20260806 19:35:42.838121 18820 raft_consensus.cc:2781] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:42.840528 18452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3a1491edbea45658e77beafec99bb2c" candidate_uuid: "7e2b53951c714d069800b3d40fdc5dbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" is_pre_election: true
I20260806 19:35:42.840695 18454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3a1491edbea45658e77beafec99bb2c" candidate_uuid: "7e2b53951c714d069800b3d40fdc5dbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8"
I20260806 19:35:42.840967 18452 raft_consensus.cc:2454] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e2b53951c714d069800b3d40fdc5dbc in term 0.
I20260806 19:35:42.847398 18820 raft_consensus.cc:686] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [term 1 LEADER]: Becoming Leader. State: Replica: 7e2b53951c714d069800b3d40fdc5dbc, State: Running, Role: LEADER
I20260806 19:35:42.847939 18820 consensus_queue.cc:237] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } } peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } }
I20260806 19:35:42.852170 18287 catalog_manager.cc:5495] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e2b53951c714d069800b3d40fdc5dbc (127.10.25.2). New cstate: current_term: 1 leader_uuid: "7e2b53951c714d069800b3d40fdc5dbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e2b53951c714d069800b3d40fdc5dbc" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 44195 } health_report { overall_health: HEALTHY } } }
I20260806 19:35:42.899223 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.891580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40114 (local address 127.10.25.2:44195)
0806 19:35:42.891845 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:42.891853 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:42.891877 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:35:42.892004 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:42.892010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:42.892131 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:42.892328 (+ 197us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:35:42.892732 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.894106 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.896113 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.896847 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.897002 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.897592 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:35:42.898408 (+ 816us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:35:42.898585 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:42.898772 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 19:35:42.898878 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 19:35:42.921156 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.912620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:34493 (local address 127.10.25.2:40703)
0806 19:35:42.913407 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:42.913435 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:42.913785 (+ 350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:42.914261 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:42.914265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:42.914276 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:35:42.914550 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:42.914558 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.916372 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.916377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:42.917725 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:42.917741 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.919132 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.919143 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:42.919644 (+ 501us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.919760 (+ 116us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:35:42.920508 (+ 748us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:35:42.920526 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 19:35:42.920617 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 19:35:42.920779 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":520,"threads_started":1}
I20260806 19:35:42.921156 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.912952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:40703 (local address 127.10.25.3:34493)
0806 19:35:42.913134 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:42.913139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:42.913560 (+ 421us) server_negotiation.cc:407] Connection header received
0806 19:35:42.913983 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:42.913990 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:42.914117 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:42.914227 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:35:42.914789 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.916062 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.917892 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.918659 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.918823 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.920058 (+ 1235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:35:42.920229 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:35:42.920376 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:42.920692 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 19:35:42.920799 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 19:35:42.932546 18735 raft_consensus.cc:1261] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d [term 1 FOLLOWER]: Refusing update from remote peer 7e2b53951c714d069800b3d40fdc5dbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:35:42.932629 18452 raft_consensus.cc:3037] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:42.933437 18914 consensus_queue.cc:1035] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa98c491ecf648d3961345af0d38c26d" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:42.936906 18452 raft_consensus.cc:1261] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 FOLLOWER]: Refusing update from remote peer 7e2b53951c714d069800b3d40fdc5dbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:35:42.938576 18820 consensus_queue.cc:1035] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c506ffdef4b2a9f98946fdbc1b0c8" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:35:42.976877 18925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.968271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:44195 (local address 127.10.25.3:36659)
0806 19:35:42.968978 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:42.968997 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:42.969120 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:42.969557 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:42.969564 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:42.969582 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 19:35:42.969872 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:42.969881 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.971978 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.971984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:42.974091 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:42.974113 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.975332 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.975339 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:42.975538 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.975584 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 19:35:42.976087 (+ 503us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 19:35:42.976102 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 19:35:42.976262 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 19:35:42.976441 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":395,"threads_started":1}
I20260806 19:35:42.976882 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:42.968818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:36659 (local address 127.10.25.2:44195)
0806 19:35:42.969024 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:42.969031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:42.969054 (+ 23us) server_negotiation.cc:407] Connection header received
0806 19:35:42.969276 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:42.969282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:42.969378 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:42.969517 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:35:42.970036 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.971363 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.974326 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:42.975119 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:42.975317 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:42.975772 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:35:42.975947 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:35:42.976054 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:42.976318 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 19:35:42.976416 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 19:35:43.031293 18289 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:35:43.031620 18289 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:35:43.035871 18289 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfc9e7847d204227805df682e7d74e0c: Sending DeleteTablet for 3 replicas of tablet b3a1491edbea45658e77beafec99bb2c
I20260806 19:35:43.037181 18428 tablet_service.cc:1514] Processing DeleteTablet for tablet b3a1491edbea45658e77beafec99bb2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:43 UTC) from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:41694
I20260806 19:35:43.037550 18875 tablet_replica.cc:330] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8: stopping tablet replica
I20260806 19:35:43.037384 18714 tablet_service.cc:1514] Processing DeleteTablet for tablet b3a1491edbea45658e77beafec99bb2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:43 UTC) from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:49346
I20260806 19:35:43.037775 18875 raft_consensus.cc:2229] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:43.037937 18877 tablet_replica.cc:330] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d: stopping tablet replica
I20260806 19:35:43.038009 18875 raft_consensus.cc:2258] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:43.038156 18877 raft_consensus.cc:2229] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:35:43.038370 18877 raft_consensus.cc:2258] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:43.038571 18572 tablet_service.cc:1514] Processing DeleteTablet for tablet b3a1491edbea45658e77beafec99bb2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:35:43 UTC) from {username='oryx', principal='oryx/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:58936
I20260806 19:35:43.039016 18876 tablet_replica.cc:330] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc: stopping tablet replica
I20260806 19:35:43.039232 18876 raft_consensus.cc:2229] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:35:43.039800 18876 raft_consensus.cc:2258] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:35:43.041206 18876 ts_tablet_manager.cc:1905] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:43.041208 18875 ts_tablet_manager.cc:1905] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:43.046948 18877 ts_tablet_manager.cc:1905] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:35:43.051331 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 18361
I20260806 19:35:43.053068 18876 ts_tablet_manager.cc:1918] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:35:43.053361 18876 log.cc:1196] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/b3a1491edbea45658e77beafec99bb2c
I20260806 19:35:43.054009 18876 ts_tablet_manager.cc:1939] T b3a1491edbea45658e77beafec99bb2c P 7e2b53951c714d069800b3d40fdc5dbc: Deleting consensus metadata
I20260806 19:35:43.055441 18273 catalog_manager.cc:4841] TS 7e2b53951c714d069800b3d40fdc5dbc (127.10.25.2:44195): tablet b3a1491edbea45658e77beafec99bb2c (table test-table [id=58c8e689c733492bb5a32e9c8f8fa4f3]) successfully deleted
I20260806 19:35:43.055222 18877 ts_tablet_manager.cc:1918] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:35:43.055867 18877 log.cc:1196] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/b3a1491edbea45658e77beafec99bb2c
I20260806 19:35:43.056295 18877 ts_tablet_manager.cc:1939] T b3a1491edbea45658e77beafec99bb2c P aa98c491ecf648d3961345af0d38c26d: Deleting consensus metadata
I20260806 19:35:43.057560 18274 catalog_manager.cc:4841] TS aa98c491ecf648d3961345af0d38c26d (127.10.25.3:34493): tablet b3a1491edbea45658e77beafec99bb2c (table test-table [id=58c8e689c733492bb5a32e9c8f8fa4f3]) successfully deleted
I20260806 19:35:43.059031 18875 ts_tablet_manager.cc:1918] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 19:35:43.059235 18875 log.cc:1196] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/b3a1491edbea45658e77beafec99bb2c
I20260806 19:35:43.059842 18875 ts_tablet_manager.cc:1939] T b3a1491edbea45658e77beafec99bb2c P 245c506ffdef4b2a9f98946fdbc1b0c8: Deleting consensus metadata
I20260806 19:35:43.061731 18274 catalog_manager.cc:4841] TS 245c506ffdef4b2a9f98946fdbc1b0c8 (127.10.25.1:32855): tablet b3a1491edbea45658e77beafec99bb2c (table test-table [id=58c8e689c733492bb5a32e9c8f8fa4f3]) successfully deleted
Aug 06 19:35:43 dist-test-slave-ss4n krb5kdc[18243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.25.1@KRBTEST.COM
I20260806 19:35:43.079144 18904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:43.052681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36080 (local address 127.10.25.1:32855)
0806 19:35:43.052940 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:43.052948 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:43.062781 (+ 9833us) server_negotiation.cc:407] Connection header received
0806 19:35:43.063061 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:43.063066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:43.063156 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:43.063276 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:43.064620 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:43.065408 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:43.066786 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:43.067249 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:43.067405 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:43.072142 (+ 4737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:43.072169 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:43.072196 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:43.072260 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:43.075795 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:43.076627 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:43.076634 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:43.076637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:43.076795 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:43.077177 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:43.077183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:43.077186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:43.077704 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:43.078044 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:43.078405 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 19:35:43.078665 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 19:35:43.079919 18495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:43.219600 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 18361
I20260806 19:35:43.244976 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 18504
Aug 06 19:35:43 dist-test-slave-ss4n krb5kdc[18243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.25.2@KRBTEST.COM
I20260806 19:35:43.263799 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:43.245859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40118 (local address 127.10.25.2:44195)
0806 19:35:43.246040 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:43.246048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:43.246093 (+ 45us) server_negotiation.cc:407] Connection header received
0806 19:35:43.246329 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:43.246335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:43.246414 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:43.246526 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:43.247828 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:43.248599 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:43.250143 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:43.250637 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:43.250785 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:43.255484 (+ 4699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:43.255507 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:43.255529 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:43.255582 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:43.259886 (+ 4304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:43.260797 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:43.260807 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:43.260811 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:43.260987 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:43.261852 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:43.261857 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:43.261861 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:43.262434 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:43.262781 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:43.263160 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 19:35:43.263415 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 19:35:43.264647 18638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:43.397784 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 18504
I20260806 19:35:43.422535 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 18647
I20260806 19:35:43.423370 18931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:43.422436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:32855 (local address 127.10.25.3:49409)
0806 19:35:43.422987 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:43.423115 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":287,"threads_started":1}
W20260806 19:35:43.423931 18665 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.25.1:32855: connect: Connection refused (error 111)
I20260806 19:35:43.425684 18931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:43.425171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:32855 (local address 127.10.25.3:55477)
0806 19:35:43.425347 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:43.425420 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 19:35:43.425983 18665 consensus_peers.cc:487] T e8d77cae0781417c85c67addae80ad5a P aa98c491ecf648d3961345af0d38c26d -> Peer 245c506ffdef4b2a9f98946fdbc1b0c8 (127.10.25.1:32855): Couldn't send request to peer 245c506ffdef4b2a9f98946fdbc1b0c8. Status: Network error: Client connection negotiation failed: client connection to 127.10.25.1:32855: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Aug 06 19:35:43 dist-test-slave-ss4n krb5kdc[18243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.25.3@KRBTEST.COM
I20260806 19:35:43.441439 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:43.423486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48540 (local address 127.10.25.3:34493)
0806 19:35:43.423944 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:43.423949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:43.423964 (+ 15us) server_negotiation.cc:407] Connection header received
0806 19:35:43.424013 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:43.424016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:43.424116 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:43.424250 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:43.425691 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:43.426651 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:43.427962 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:43.428405 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:43.428557 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:43.433628 (+ 5071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:43.433660 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:43.433691 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:43.433770 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:43.438223 (+ 4453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:43.439191 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:43.439200 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:43.439202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:43.439388 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:43.439871 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:43.439875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:43.439878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:43.440273 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:43.440557 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:43.440929 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 19:35:43.441119 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":233,"threads_started":1}
I20260806 19:35:43.442281 18782 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:43.603555 18931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:43.602907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:44195 (local address 127.10.25.3:33057)
0806 19:35:43.603152 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:43.603298 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.2:44195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 19:35:43.603547 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 18647
I20260806 19:35:43.634191 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 18256
I20260806 19:35:43.635764 18330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:43.761628 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 18256
2026-08-06T19:35:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7098 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:35:43 dist-test-slave-ss4n krb5kdc[18938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:35:43 dist-test-slave-ss4n krb5kdc[18938](info): set up 2 sockets
Aug 06 19:35:43 dist-test-slave-ss4n krb5kdc[18938](info): commencing operation
krb5kdc: starting...
W20260806 19:35:44.991034 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.005s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:35:45 dist-test-slave-ss4n krb5kdc[18938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:35:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:35:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:35:45.282900 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:42221
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:37567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:42221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:45.661756 18952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:45.662086 18952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:45.662201 18952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:45.670176 18952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:35:45.670313 18952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:45.670360 18952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:45.670419 18952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:35:45.670482 18952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:35:45.683629 18952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:37567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:42221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:42221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:45.685087 18952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:45.686578 18952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:45.700785 18957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:45.700760 18958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:45.701151 18960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:45.701777 18952 server_base.cc:1034] running on GCE node
I20260806 19:35:45.702718 18952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:45.704389 18952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:45.705664 18952 hybrid_clock.cc:648] HybridClock initialized: now 1786044945705597 us; error 50 us; skew 500 ppm
I20260806 19:35:45.706121 18952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T19:35:45Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2025 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:35:45 dist-test-slave-ss4n krb5kdc[18966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:35:45 dist-test-slave-ss4n krb5kdc[18966](info): set up 2 sockets
Aug 06 19:35:45 dist-test-slave-ss4n krb5kdc[18966](info): commencing operation
krb5kdc: starting...
W20260806 19:35:47.020160 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:35:47 dist-test-slave-ss4n krb5kdc[18966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:35:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:35:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:35:47.352272 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:44085
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:44085
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:47.751982 18980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:47.752332 18980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:47.752513 18980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:47.760811 18980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:35:47.760986 18980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:47.761041 18980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:47.761143 18980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:35:47.761238 18980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:35:47.773833 18980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:44085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:44085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:47.775480 18980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:47.776975 18980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:47.792857 18985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:47.792991 18988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:47.793272 18986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:47.795073 18980 server_base.cc:1034] running on GCE node
I20260806 19:35:47.796528 18980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:47.798497 18980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:47.799898 18980 hybrid_clock.cc:648] HybridClock initialized: now 1786044947799816 us; error 98 us; skew 500 ppm
I20260806 19:35:47.800684 18980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:47 dist-test-slave-ss4n krb5kdc[18966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044947, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:47.805927 18980 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:35:47.808017 18980 webserver.cc:460] Webserver started at http://127.10.25.62:41205/ using document root <none> and password file <none>
I20260806 19:35:47.808846 18980 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:47.809023 18980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:47.809373 18980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:47.812799 18980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "6c9ea4d338684e389be59ec9f96314aa"
format_stamp: "Formatted at 2026-08-06 19:35:47 on dist-test-slave-ss4n"
server_key: "596166b597f1346eba6c921387d9851c"
server_key_iv: "1bc54686f3aa0319b04c21dbbc277b08"
server_key_version: "encryptionkey@0"
I20260806 19:35:47.813813 18980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "6c9ea4d338684e389be59ec9f96314aa"
format_stamp: "Formatted at 2026-08-06 19:35:47 on dist-test-slave-ss4n"
server_key: "596166b597f1346eba6c921387d9851c"
server_key_iv: "1bc54686f3aa0319b04c21dbbc277b08"
server_key_version: "encryptionkey@0"
I20260806 19:35:47.821238 18980 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.004s
I20260806 19:35:47.826429 18995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:47.828357 18980 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260806 19:35:47.828853 18980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "6c9ea4d338684e389be59ec9f96314aa"
format_stamp: "Formatted at 2026-08-06 19:35:47 on dist-test-slave-ss4n"
server_key: "596166b597f1346eba6c921387d9851c"
server_key_iv: "1bc54686f3aa0319b04c21dbbc277b08"
server_key_version: "encryptionkey@0"
I20260806 19:35:47.829416 18980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:47.860100 18980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:47.866544 18980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:47.867039 18980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:47.891280 18980 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:44085
I20260806 19:35:47.891302 19047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:44085 every 8 connection(s)
I20260806 19:35:47.893054 18980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:35:47.899700 19048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:35:47.900074 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 18980
I20260806 19:35:47.900345 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:35:47.901048 10340 external_mini_cluster.cc:1428] Setting key 734b4c9fbddb1e449046b839adf3af36
I20260806 19:35:47.916915 19048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa: Bootstrap starting.
Aug 06 19:35:47 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:35:47.922084 19048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa: Neither blocks nor log segments found. Creating new log.
I20260806 19:35:47.923779 19048 log.cc:824] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:47.929364 19048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa: No bootstrap required, opened a new log
I20260806 19:35:47.932542 19051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:47.903947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44870 (local address 127.10.25.62:44085)
0806 19:35:47.905069 (+ 1122us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:47.905084 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:47.905138 (+ 54us) server_negotiation.cc:407] Connection header received
0806 19:35:47.906967 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:47.906997 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:47.907506 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:47.908017 (+ 511us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:47.909928 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:47.911284 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:47.912694 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:47.913637 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:47.913854 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:47.920557 (+ 6703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:47.920571 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:47.920595 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:47.920665 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:47.925892 (+ 5227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:47.927252 (+ 1360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:47.927263 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:47.927274 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:47.927581 (+ 307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:47.928453 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:47.928460 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:47.928462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:47.929433 (+ 971us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:47.929889 (+ 456us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:47.930684 (+ 795us) server_negotiation.cc:299] Negotiation successful
0806 19:35:47.931254 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":818,"thread_start_us":500,"threads_started":1}
I20260806 19:35:47.941984 19048 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9ea4d338684e389be59ec9f96314aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44085 } }
I20260806 19:35:47.942562 19048 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:35:47.942698 19048 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c9ea4d338684e389be59ec9f96314aa, State: Initialized, Role: FOLLOWER
I20260806 19:35:47.943451 19048 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9ea4d338684e389be59ec9f96314aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44085 } }
I20260806 19:35:47.943746 19048 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:35:47.943914 19048 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:35:47.944123 19048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:35:47.947603 19048 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9ea4d338684e389be59ec9f96314aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44085 } }
I20260806 19:35:47.948249 19048 leader_election.cc:302] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c9ea4d338684e389be59ec9f96314aa; no voters:
I20260806 19:35:47.949002 19048 leader_election.cc:288] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:35:47.949261 19053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:35:47.950731 19053 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 1 LEADER]: Becoming Leader. State: Replica: 6c9ea4d338684e389be59ec9f96314aa, State: Running, Role: LEADER
I20260806 19:35:47.951346 19053 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9ea4d338684e389be59ec9f96314aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44085 } }
I20260806 19:35:47.951638 19048 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:35:47.954471 19055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c9ea4d338684e389be59ec9f96314aa. Latest consensus state: current_term: 1 leader_uuid: "6c9ea4d338684e389be59ec9f96314aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9ea4d338684e389be59ec9f96314aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44085 } } }
I20260806 19:35:47.954921 19055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:47.955528 19054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c9ea4d338684e389be59ec9f96314aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9ea4d338684e389be59ec9f96314aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44085 } } }
I20260806 19:35:47.956019 19054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:47.957193 19062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:35:47.965713 19062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:35:47.976729 19062 catalog_manager.cc:1347] Generated new cluster ID: 178f3cbcc5b6487098b409abb8e46ca9
I20260806 19:35:47.976874 19062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:35:48.002635 19062 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:35:48.004650 19062 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:35:48.014271 19062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa: Generated new TSK 0
I20260806 19:35:48.015625 19062 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:35:48.095108 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44085
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:48.506916 19074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:48.507256 19074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:48.507426 19074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:48.516947 19074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:48.517131 19074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:48.517369 19074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:35:48.530175 19074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:48.531966 19074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:48.533629 19074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:48.549672 19082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:48.549998 19079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:48.549918 19080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:48.550588 19074 server_base.cc:1034] running on GCE node
I20260806 19:35:48.551276 19074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:48.552702 19074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:48.554373 19074 hybrid_clock.cc:648] HybridClock initialized: now 1786044948554263 us; error 83 us; skew 500 ppm
I20260806 19:35:48.554857 19074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:48 dist-test-slave-ss4n krb5kdc[18966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044948, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:48.558717 19074 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:35:48.560556 19074 webserver.cc:460] Webserver started at http://127.10.25.1:33725/ using document root <none> and password file <none>
I20260806 19:35:48.561378 19074 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:48.561573 19074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:48.561939 19074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:48.565282 19074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "ec065d265dbf499fab4b8cc2a73ebbed"
format_stamp: "Formatted at 2026-08-06 19:35:48 on dist-test-slave-ss4n"
server_key: "40cb9c9d4c29113c20747e597003cfb0"
server_key_iv: "6f98710cd8362ce5c229025a2a78f7ae"
server_key_version: "encryptionkey@0"
I20260806 19:35:48.566322 19074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "ec065d265dbf499fab4b8cc2a73ebbed"
format_stamp: "Formatted at 2026-08-06 19:35:48 on dist-test-slave-ss4n"
server_key: "40cb9c9d4c29113c20747e597003cfb0"
server_key_iv: "6f98710cd8362ce5c229025a2a78f7ae"
server_key_version: "encryptionkey@0"
I20260806 19:35:48.572369 19074 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260806 19:35:48.577144 19090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:48.579350 19074 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 19:35:48.579702 19074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "ec065d265dbf499fab4b8cc2a73ebbed"
format_stamp: "Formatted at 2026-08-06 19:35:48 on dist-test-slave-ss4n"
server_key: "40cb9c9d4c29113c20747e597003cfb0"
server_key_iv: "6f98710cd8362ce5c229025a2a78f7ae"
server_key_version: "encryptionkey@0"
I20260806 19:35:48.579975 19074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:48.619019 19074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:48.625870 19074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:48.626278 19074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:48.627526 19074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:48.629192 19074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:48.629316 19074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:48.629505 19074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:48.629592 19074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 19:35:48.675104 19074 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:36743
I20260806 19:35:48.675213 19204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:36743 every 8 connection(s)
I20260806 19:35:48.676700 19074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:35:48.678092 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 19074
I20260806 19:35:48.678328 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:35:48.679322 10340 external_mini_cluster.cc:1428] Setting key 6ae1b6b766033b160a5e54735a29e59a
Aug 06 19:35:48 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044948, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:35:48.706478 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:48.680682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:40653 (local address 127.10.25.62:44085)
0806 19:35:48.681367 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:48.681374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:48.683094 (+ 1720us) server_negotiation.cc:407] Connection header received
0806 19:35:48.686132 (+ 3038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:48.686143 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:48.686313 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:48.686473 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:48.688521 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:48.689885 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:48.691958 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:48.692963 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:48.693179 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:48.698327 (+ 5148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:48.698333 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:48.698337 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:48.698378 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:48.701918 (+ 3540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:48.703189 (+ 1271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:48.703197 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:48.703203 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:48.703419 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:48.704002 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:48.704009 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:48.704013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:48.704408 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:48.704605 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:48.705650 (+ 1045us) server_negotiation.cc:299] Negotiation successful
0806 19:35:48.705992 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":627,"threads_started":1}
I20260806 19:35:48.708266 19208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:48.681587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:40653)
0806 19:35:48.682860 (+ 1273us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:48.682928 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:48.685644 (+ 2716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:48.686818 (+ 1174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:48.686829 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:48.687536 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:48.688233 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:48.688260 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:48.690211 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:48.690216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:48.691695 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:48.691716 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:48.693441 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:48.693457 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:48.693885 (+ 428us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:48.694928 (+ 1043us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:48.694958 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:48.698001 (+ 3043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:48.702255 (+ 4254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:48.702268 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:48.702291 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:48.703004 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:48.703571 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:48.703584 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:48.703588 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:48.703858 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:48.704567 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:48.704579 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:48.705115 (+ 536us) client_negotiation.cc:769] Sending connection context
0806 19:35:48.705485 (+ 370us) client_negotiation.cc:240] Negotiation successful
0806 19:35:48.706308 (+ 823us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":863,"thread_start_us":546,"threads_started":1}
I20260806 19:35:48.710793 19205 heartbeater.cc:347] Connected to a master server at 127.10.25.62:44085
I20260806 19:35:48.711215 19205 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
I20260806 19:35:48.712142 19205 heartbeater.cc:511] Master 127.10.25.62:44085 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
I20260806 19:35:48.715986 19012 ts_manager.cc:194] Registered new tserver with Master: ec065d265dbf499fab4b8cc2a73ebbed (127.10.25.1:36743)
I20260806 19:35:48.718637 19012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:40653
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:35:48.757354 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44085
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:49.141260 19211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:49.141724 19211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:49.141903 19211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:49.151057 19211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:49.151252 19211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:49.151520 19211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:35:49.162757 19211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:49.164376 19211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:49.165875 19211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:49.182791 19216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:49.182753 19219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:49.182755 19217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:49.183725 19211 server_base.cc:1034] running on GCE node
I20260806 19:35:49.184546 19211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:49.186045 19211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:49.187520 19211 hybrid_clock.cc:648] HybridClock initialized: now 1786044949187454 us; error 61 us; skew 500 ppm
I20260806 19:35:49.187973 19211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:49 dist-test-slave-ss4n krb5kdc[18966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044949, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:49.192094 19211 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:35:49.194082 19211 webserver.cc:460] Webserver started at http://127.10.25.2:40819/ using document root <none> and password file <none>
I20260806 19:35:49.195837 19211 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:49.196054 19211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:49.196444 19211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:49.199975 19211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "581fac8ed7194f9fa4f057a4ca479730"
format_stamp: "Formatted at 2026-08-06 19:35:49 on dist-test-slave-ss4n"
server_key: "558b8d0ceef0f5ea5f8a378e9cae5c50"
server_key_iv: "15bfbfe5d8415ffc418dfb7053264aa5"
server_key_version: "encryptionkey@0"
I20260806 19:35:49.200913 19211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "581fac8ed7194f9fa4f057a4ca479730"
format_stamp: "Formatted at 2026-08-06 19:35:49 on dist-test-slave-ss4n"
server_key: "558b8d0ceef0f5ea5f8a378e9cae5c50"
server_key_iv: "15bfbfe5d8415ffc418dfb7053264aa5"
server_key_version: "encryptionkey@0"
I20260806 19:35:49.207335 19211 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 19:35:49.213172 19226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:49.215219 19211 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 19:35:49.215471 19211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "581fac8ed7194f9fa4f057a4ca479730"
format_stamp: "Formatted at 2026-08-06 19:35:49 on dist-test-slave-ss4n"
server_key: "558b8d0ceef0f5ea5f8a378e9cae5c50"
server_key_iv: "15bfbfe5d8415ffc418dfb7053264aa5"
server_key_version: "encryptionkey@0"
I20260806 19:35:49.215764 19211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:49.243201 19211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:49.249635 19211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:49.250001 19211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:49.251492 19211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:49.253382 19211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:49.253471 19211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:49.253674 19211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:49.253729 19211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:49.296923 19211 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:33801
I20260806 19:35:49.296967 19340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:33801 every 8 connection(s)
I20260806 19:35:49.298496 19211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:35:49.299330 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 19211
I20260806 19:35:49.299784 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:35:49.300836 10340 external_mini_cluster.cc:1428] Setting key 7fa1a726c4dadfc075a01da4b684767a
Aug 06 19:35:49 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044949, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:35:49.330030 19343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:49.302728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:55091 (local address 127.10.25.62:44085)
0806 19:35:49.303305 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:49.303313 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:49.304894 (+ 1581us) server_negotiation.cc:407] Connection header received
0806 19:35:49.307176 (+ 2282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:49.307185 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:49.307308 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:49.307445 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:49.310467 (+ 3022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:49.311727 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:49.313858 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:49.314642 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:49.314813 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:49.320371 (+ 5558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:49.320382 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:49.320390 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:49.320458 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:49.324933 (+ 4475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:49.326262 (+ 1329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:49.326271 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:49.326277 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:49.326482 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:49.327357 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:49.327365 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:49.327371 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:49.327880 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:49.328092 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:49.329203 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0806 19:35:49.329563 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":281,"threads_started":1}
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
I20260806 19:35:49.331204 19344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:49.303620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:55091)
0806 19:35:49.304663 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:49.304726 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:49.306801 (+ 2075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:49.308007 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:49.308023 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:49.308990 (+ 967us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:49.310064 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:49.310096 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:49.312047 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:49.312053 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:49.313648 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:49.313669 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:49.314931 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:49.314937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:49.315196 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:49.316133 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:49.316174 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:49.319954 (+ 3780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:49.325224 (+ 5270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:49.325235 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:49.325253 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:49.326049 (+ 796us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:49.326759 (+ 710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:49.326765 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:49.326769 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:49.327130 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:49.328120 (+ 990us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:49.328130 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:49.328635 (+ 505us) client_negotiation.cc:769] Sending connection context
0806 19:35:49.329022 (+ 387us) client_negotiation.cc:240] Negotiation successful
0806 19:35:49.329684 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":430,"threads_started":1}
I20260806 19:35:49.333948 19341 heartbeater.cc:347] Connected to a master server at 127.10.25.62:44085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
I20260806 19:35:49.334417 19341 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:49.335335 19341 heartbeater.cc:511] Master 127.10.25.62:44085 requested a full tablet report, sending...
I20260806 19:35:49.337275 19012 ts_manager.cc:194] Registered new tserver with Master: 581fac8ed7194f9fa4f057a4ca479730 (127.10.25.2:33801)
I20260806 19:35:49.339002 19012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:55091
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:35:49.375202 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44085
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:35:49.722836 19205 heartbeater.cc:503] Master 127.10.25.62:44085 was elected leader, sending a full tablet report...
W20260806 19:35:49.777112 19348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:49.777429 19348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:49.777642 19348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:49.785657 19348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:49.785789 19348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:49.786001 19348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:35:49.797272 19348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:49.798897 19348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:49.800379 19348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:49.819818 19354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:49.820013 19356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:49.819898 19353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:49.822573 19348 server_base.cc:1034] running on GCE node
I20260806 19:35:49.823601 19348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:49.825029 19348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:49.826392 19348 hybrid_clock.cc:648] HybridClock initialized: now 1786044949826317 us; error 68 us; skew 500 ppm
I20260806 19:35:49.826802 19348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:49 dist-test-slave-ss4n krb5kdc[18966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044949, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:49.832110 19348 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:35:49.834309 19348 webserver.cc:460] Webserver started at http://127.10.25.3:35463/ using document root <none> and password file <none>
I20260806 19:35:49.835188 19348 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:49.835348 19348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:49.835736 19348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:35:49.839607 19348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "2d3d73453b024d5a8f7629600f02c0e9"
format_stamp: "Formatted at 2026-08-06 19:35:49 on dist-test-slave-ss4n"
server_key: "e325bac356caa48a7ea0c8777a432a5b"
server_key_iv: "0ff7a91b84b787696d7614a2b112c665"
server_key_version: "encryptionkey@0"
I20260806 19:35:49.840611 19348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "2d3d73453b024d5a8f7629600f02c0e9"
format_stamp: "Formatted at 2026-08-06 19:35:49 on dist-test-slave-ss4n"
server_key: "e325bac356caa48a7ea0c8777a432a5b"
server_key_iv: "0ff7a91b84b787696d7614a2b112c665"
server_key_version: "encryptionkey@0"
I20260806 19:35:49.848198 19348 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 19:35:49.853981 19363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:49.855682 19348 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:35:49.855957 19348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "2d3d73453b024d5a8f7629600f02c0e9"
format_stamp: "Formatted at 2026-08-06 19:35:49 on dist-test-slave-ss4n"
server_key: "e325bac356caa48a7ea0c8777a432a5b"
server_key_iv: "0ff7a91b84b787696d7614a2b112c665"
server_key_version: "encryptionkey@0"
I20260806 19:35:49.856204 19348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:49.908620 19348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:49.915946 19348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:49.916415 19348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:49.918310 19348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:49.926575 19348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:49.926715 19348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:49.926860 19348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:49.926930 19348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:49.971701 19348 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:37507
I20260806 19:35:49.971746 19477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:37507 every 8 connection(s)
I20260806 19:35:49.973292 19348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:35:49.982856 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 19348
I20260806 19:35:49.983072 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:35:49.983701 10340 external_mini_cluster.cc:1428] Setting key c90f90e97ce08ea0548ae25d50690071
Aug 06 19:35:49 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044949, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:35:50.002079 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:49.976785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:46689 (local address 127.10.25.62:44085)
0806 19:35:49.977366 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:49.977372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:49.978636 (+ 1264us) server_negotiation.cc:407] Connection header received
0806 19:35:49.981085 (+ 2449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:49.981092 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:49.981216 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:49.981357 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:49.983821 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:49.985135 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:49.987396 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:49.987942 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:49.988128 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:49.993144 (+ 5016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:49.993151 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:49.993156 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:49.993203 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:49.996866 (+ 3663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:49.998340 (+ 1474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:49.998347 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:49.998350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:49.998582 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:49.999555 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:49.999562 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:49.999565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:49.999993 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:50.000167 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:50.001290 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0806 19:35:50.001700 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":287,"threads_started":1}
I20260806 19:35:50.003747 19481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:49.977392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:46689)
0806 19:35:49.978388 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:49.978435 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:49.980698 (+ 2263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:49.981703 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:49.981714 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:49.982475 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:49.983481 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:49.983513 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:49.985447 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:49.985452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:49.987156 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:49.987178 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:49.988263 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:49.988272 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:49.988671 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:49.989697 (+ 1026us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:49.989726 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:49.992651 (+ 2925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:35:49.997233 (+ 4582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:49.997244 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:49.997267 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:49.998000 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:49.998762 (+ 762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:35:49.998766 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:35:49.998769 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:35:49.999327 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:35:50.000215 (+ 888us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:35:50.000227 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:35:50.000718 (+ 491us) client_negotiation.cc:769] Sending connection context
0806 19:35:50.001051 (+ 333us) client_negotiation.cc:240] Negotiation successful
0806 19:35:50.001889 (+ 838us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":392,"threads_started":1}
I20260806 19:35:50.006239 19478 heartbeater.cc:347] Connected to a master server at 127.10.25.62:44085
I20260806 19:35:50.006726 19478 heartbeater.cc:464] Registering TS with master...
I20260806 19:35:50.007818 19478 heartbeater.cc:511] Master 127.10.25.62:44085 requested a full tablet report, sending...
I20260806 19:35:50.010114 19012 ts_manager.cc:194] Registered new tserver with Master: 2d3d73453b024d5a8f7629600f02c0e9 (127.10.25.3:37507)
I20260806 19:35:50.011708 19012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:46689
I20260806 19:35:50.016012 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.1.keytab.
Entry for principal oryx/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.1.keytab.
WARNING: no policy specified for oryx/127.10.25.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.25.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.2.keytab.
Entry for principal oryx/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.2.keytab.
WARNING: no policy specified for oryx/127.10.25.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.25.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.3.keytab.
Entry for principal oryx/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.3.keytab.
I20260806 19:35:50.214447 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 19074
Aug 06 19:35:50 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:35:50.240768 19489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:50.216136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57344 (local address 127.10.25.1:36743)
0806 19:35:50.216994 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:50.217002 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:50.217706 (+ 704us) server_negotiation.cc:407] Connection header received
0806 19:35:50.217822 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:50.217829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:50.218092 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:50.218275 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:50.220790 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:50.222268 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:50.224215 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:50.224968 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:50.225137 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:50.231180 (+ 6043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:50.231217 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:50.231249 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:50.231316 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:50.235855 (+ 4539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:50.237366 (+ 1511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:50.237378 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:50.237380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:50.237697 (+ 317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:50.238688 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:50.238695 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:50.238698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:50.239254 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:50.239573 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:50.240150 (+ 577us) server_negotiation.cc:299] Negotiation successful
0806 19:35:50.240436 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":429,"threads_started":1}
I20260806 19:35:50.243016 19200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:50.336961 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 19074
I20260806 19:35:50.344025 19341 heartbeater.cc:503] Master 127.10.25.62:44085 was elected leader, sending a full tablet report...
I20260806 19:35:50.358749 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 19211
Aug 06 19:35:50 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:35:50.383833 19491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:50.360400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56114 (local address 127.10.25.2:33801)
0806 19:35:50.361606 (+ 1206us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:50.361619 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:50.361671 (+ 52us) server_negotiation.cc:407] Connection header received
0806 19:35:50.361763 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:50.361771 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:50.362023 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:50.362247 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:50.364434 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:50.365914 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:50.367997 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:50.368725 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:50.368883 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:50.374523 (+ 5640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:50.374544 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:50.374564 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:50.374613 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:50.378288 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:50.379829 (+ 1541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:50.379845 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:50.379848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:50.380118 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:50.381033 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:50.381041 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:50.381043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:50.381730 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:50.382066 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:50.382854 (+ 788us) server_negotiation.cc:299] Negotiation successful
0806 19:35:50.383309 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":866,"thread_start_us":534,"threads_started":1}
I20260806 19:35:50.385746 19336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:50.478698 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 19211
I20260806 19:35:50.501384 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 19348
Aug 06 19:35:50 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:35:50.524044 19493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:50.502747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36098 (local address 127.10.25.3:37507)
0806 19:35:50.503566 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:50.503576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:50.503632 (+ 56us) server_negotiation.cc:407] Connection header received
0806 19:35:50.503719 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:50.503730 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:50.504049 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:50.504468 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:50.506334 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:50.507355 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:50.508783 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:50.509404 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:50.509620 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:50.515153 (+ 5533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:50.515200 (+ 47us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:50.515224 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:50.515291 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:50.519186 (+ 3895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:50.520399 (+ 1213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:50.520414 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:50.520417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:50.520698 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:50.521293 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:50.521299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:50.521301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:50.522099 (+ 798us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:50.522414 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:50.523146 (+ 732us) server_negotiation.cc:299] Negotiation successful
0806 19:35:50.523623 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":319,"threads_started":1}
I20260806 19:35:50.526119 19473 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:50.618686 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 19348
I20260806 19:35:50.640308 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 18980
I20260806 19:35:50.642127 19043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:35:50.754094 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 18980
I20260806 19:35:50.780910 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:44085
--webserver_interface=127.10.25.62
--webserver_port=41205
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:44085
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:35:51.166680 19496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:51.167044 19496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:51.167145 19496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:51.175171 19496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:35:51.175341 19496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:51.175395 19496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:51.175463 19496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:35:51.175525 19496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:35:51.187083 19496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:44085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:44085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=41205
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:51.188514 19496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:51.189913 19496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:51.203524 19502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:51.203558 19503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:51.203472 19505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:51.204397 19496 server_base.cc:1034] running on GCE node
I20260806 19:35:51.205405 19496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:51.207386 19496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:51.210952 19496 hybrid_clock.cc:648] HybridClock initialized: now 1786044951210834 us; error 109 us; skew 500 ppm
I20260806 19:35:51.211545 19496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:51 dist-test-slave-ss4n krb5kdc[18966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044951, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:51.221028 19496 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:35:51.223909 19496 webserver.cc:460] Webserver started at http://127.10.25.62:41205/ using document root <none> and password file <none>
I20260806 19:35:51.225095 19496 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:51.225349 19496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:51.231930 19496 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 19:35:51.235489 19512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:51.237850 19496 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:35:51.238088 19496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "6c9ea4d338684e389be59ec9f96314aa"
format_stamp: "Formatted at 2026-08-06 19:35:47 on dist-test-slave-ss4n"
server_key: "596166b597f1346eba6c921387d9851c"
server_key_iv: "1bc54686f3aa0319b04c21dbbc277b08"
server_key_version: "encryptionkey@0"
I20260806 19:35:51.239214 19496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:51.261829 19496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:51.267587 19496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:51.267925 19496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:51.291009 19496 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:44085
I20260806 19:35:51.291023 19564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:44085 every 8 connection(s)
I20260806 19:35:51.292829 19496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:35:51.295400 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 19496
I20260806 19:35:51.296701 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:36743
--local_ip_for_outbound_sockets=127.10.25.1
--tserver_master_addrs=127.10.25.62:44085
--webserver_port=33725
--webserver_interface=127.10.25.1
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.25.1
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:35:51.305210 19565 sys_catalog.cc:263] Verifying existing consensus state
I20260806 19:35:51.309968 19565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa: Bootstrap starting.
I20260806 19:35:51.333178 19565 log.cc:824] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa: Log is configured to *not* fsync() on all Append() calls
I20260806 19:35:51.344018 19565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:35:51.344660 19565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa: Bootstrap complete.
I20260806 19:35:51.354816 19565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9ea4d338684e389be59ec9f96314aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44085 } }
I20260806 19:35:51.355844 19565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c9ea4d338684e389be59ec9f96314aa, State: Initialized, Role: FOLLOWER
I20260806 19:35:51.356575 19565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9ea4d338684e389be59ec9f96314aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44085 } }
I20260806 19:35:51.356873 19565 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:35:51.357028 19565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:35:51.357227 19565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 1 FOLLOWER]: Advancing to term 2
I20260806 19:35:51.361871 19565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9ea4d338684e389be59ec9f96314aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44085 } }
I20260806 19:35:51.362553 19565 leader_election.cc:302] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c9ea4d338684e389be59ec9f96314aa; no voters:
I20260806 19:35:51.363317 19565 leader_election.cc:288] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 19:35:51.363487 19569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 2 FOLLOWER]: Leader election won for term 2
I20260806 19:35:51.365031 19569 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [term 2 LEADER]: Becoming Leader. State: Replica: 6c9ea4d338684e389be59ec9f96314aa, State: Running, Role: LEADER
I20260806 19:35:51.365830 19569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9ea4d338684e389be59ec9f96314aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44085 } }
I20260806 19:35:51.366142 19565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:35:51.368789 19571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c9ea4d338684e389be59ec9f96314aa. Latest consensus state: current_term: 2 leader_uuid: "6c9ea4d338684e389be59ec9f96314aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9ea4d338684e389be59ec9f96314aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44085 } } }
I20260806 19:35:51.369318 19571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:51.369980 19570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6c9ea4d338684e389be59ec9f96314aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9ea4d338684e389be59ec9f96314aa" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44085 } } }
I20260806 19:35:51.370630 19570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa [sys.catalog]: This master's current role is: LEADER
I20260806 19:35:51.372323 19578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:35:51.383723 19578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:35:51.385412 19578 catalog_manager.cc:1259] Loaded cluster ID: 178f3cbcc5b6487098b409abb8e46ca9
I20260806 19:35:51.385568 19578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:35:51.389993 19578 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:35:51.391824 19578 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6c9ea4d338684e389be59ec9f96314aa: Loaded TSK: 0
I20260806 19:35:51.393682 19578 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 19:35:51.697599 19567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:51.697950 19567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:51.698108 19567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:51.706214 19567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:51.706354 19567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:51.706549 19567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:35:51.717373 19567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.1.keytab
--principal=oryx/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:36743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=33725
--tserver_master_addrs=127.10.25.62:44085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:51.719094 19567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:51.720551 19567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:51.736794 19592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:51.736459 19593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:51.737439 19567 server_base.cc:1034] running on GCE node
W20260806 19:35:51.737628 19595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:51.738584 19567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:51.739991 19567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:51.741290 19567 hybrid_clock.cc:648] HybridClock initialized: now 1786044951741211 us; error 74 us; skew 500 ppm
I20260806 19:35:51.741771 19567 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:51 dist-test-slave-ss4n krb5kdc[18966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044951, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:51.746781 19567 init.cc:377] Logged in from keytab as oryx/127.10.25.1@KRBTEST.COM (short username oryx)
I20260806 19:35:51.749166 19567 webserver.cc:460] Webserver started at http://127.10.25.1:33725/ using document root <none> and password file <none>
I20260806 19:35:51.750211 19567 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:51.750391 19567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:51.758298 19567 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 19:35:51.762398 19604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:51.764691 19567 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:35:51.764986 19567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "ec065d265dbf499fab4b8cc2a73ebbed"
format_stamp: "Formatted at 2026-08-06 19:35:48 on dist-test-slave-ss4n"
server_key: "40cb9c9d4c29113c20747e597003cfb0"
server_key_iv: "6f98710cd8362ce5c229025a2a78f7ae"
server_key_version: "encryptionkey@0"
I20260806 19:35:51.766683 19567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:51.797605 19567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:51.805114 19567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:51.805521 19567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:51.806958 19567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:51.809784 19567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:51.809930 19567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:51.810072 19567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:51.810212 19567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:51.853721 19567 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:36743
I20260806 19:35:51.853801 19719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:36743 every 8 connection(s)
I20260806 19:35:51.855459 19567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:35:51.862263 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 19567
I20260806 19:35:51.863183 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:33801
--local_ip_for_outbound_sockets=127.10.25.2
--tserver_master_addrs=127.10.25.62:44085
--webserver_port=40819
--webserver_interface=127.10.25.2
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.25.2
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:35:51 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:51 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:51.882899 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:51.859427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:45493)
0806 19:35:51.860565 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:51.860625 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:51.862673 (+ 2048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:51.864754 (+ 2081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:51.864763 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:51.865627 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:51.866366 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:51.866390 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:51.869464 (+ 3074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:51.869471 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:51.871631 (+ 2160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:51.871688 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:51.873940 (+ 2252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:51.873955 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:51.874534 (+ 579us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:51.875842 (+ 1308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:51.875887 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:51.881346 (+ 5459us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":809,"thread_start_us":578,"threads_started":1}
W20260806 19:35:51.882926 19722 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:51.859388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:45493 (local address 127.10.25.62:44085)
0806 19:35:51.860225 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:51.860237 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:51.860852 (+ 615us) server_negotiation.cc:407] Connection header received
0806 19:35:51.863437 (+ 2585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:51.863464 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:51.864067 (+ 603us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:51.864579 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:51.866684 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:51.869051 (+ 2367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:51.871934 (+ 2883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:51.873382 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:51.873744 (+ 362us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:51.881848 (+ 8104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:45493: BlockingRecv error: recv got EOF from 127.10.25.1:45493 (error 108)
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":340,"threads_started":1}
W20260806 19:35:51.883898 19720 heartbeater.cc:644] Failed to heartbeat to 127.10.25.62:44085 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.25.62:44085: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Aug 06 19:35:51 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:51 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:51.898263 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:51.884619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:42747)
0806 19:35:51.884905 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:51.884932 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:51.885176 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:51.885739 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:51.885747 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:51.886317 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:51.886606 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:51.886626 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:51.887873 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:51.887883 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:51.889958 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:51.889985 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:51.890893 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:51.890903 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:51.891163 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:51.892093 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:51.892131 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:51.897915 (+ 5784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260806 19:35:51.899125 19722 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:51.884774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:42747 (local address 127.10.25.62:44085)
0806 19:35:51.884931 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:51.884935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:51.885027 (+ 92us) server_negotiation.cc:407] Connection header received
0806 19:35:51.885401 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:51.885406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:51.885477 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:51.885701 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:51.886828 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:51.887678 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:51.890200 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:51.890680 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:51.890803 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:51.898866 (+ 8063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:42747: BlockingRecv error: recv got EOF from 127.10.25.1:42747 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 19:35:51 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:51 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:51.911159 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:51.899428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:57197)
0806 19:35:51.899671 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:51.899694 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:51.899825 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:51.900179 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:51.900184 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:51.900669 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:51.900906 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:51.900919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:51.902249 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:51.902257 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:51.904019 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:51.904041 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:51.904991 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:51.904999 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:51.905250 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:51.906002 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:51.906030 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:51.910774 (+ 4744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 19:35:51.911464 19722 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:51.899599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:57197 (local address 127.10.25.62:44085)
0806 19:35:51.899764 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:51.899769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:51.899789 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:35:51.899955 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:51.899959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:51.900022 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:51.900119 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:51.901130 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:51.902054 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:51.904280 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:51.904794 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:51.904924 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:51.911098 (+ 6174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:57197: BlockingRecv error: recv got EOF from 127.10.25.1:57197 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 19:35:51.911865 19720 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 19:35:52.271811 19724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:52.272181 19724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:52.272323 19724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:52.280303 19724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:52.280447 19724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:52.280627 19724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:35:52.294245 19724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.2.keytab
--principal=oryx/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:33801
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=40819
--tserver_master_addrs=127.10.25.62:44085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:52.295869 19724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:52.297410 19724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:52.313045 19729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:52.312945 19730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:52.313102 19732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:52.315256 19724 server_base.cc:1034] running on GCE node
I20260806 19:35:52.316148 19724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:52.317399 19724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:52.318747 19724 hybrid_clock.cc:648] HybridClock initialized: now 1786044952318705 us; error 50 us; skew 500 ppm
I20260806 19:35:52.319231 19724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:52 dist-test-slave-ss4n krb5kdc[18966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044952, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:35:52.324668 19724 init.cc:377] Logged in from keytab as oryx/127.10.25.2@KRBTEST.COM (short username oryx)
I20260806 19:35:52.327015 19724 webserver.cc:460] Webserver started at http://127.10.25.2:40819/ using document root <none> and password file <none>
I20260806 19:35:52.328150 19724 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:52.328316 19724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:52.336429 19724 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 19:35:52.340329 19739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:52.342461 19724 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 19:35:52.342672 19724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "581fac8ed7194f9fa4f057a4ca479730"
format_stamp: "Formatted at 2026-08-06 19:35:49 on dist-test-slave-ss4n"
server_key: "558b8d0ceef0f5ea5f8a378e9cae5c50"
server_key_iv: "15bfbfe5d8415ffc418dfb7053264aa5"
server_key_version: "encryptionkey@0"
I20260806 19:35:52.344027 19724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:52.361563 19724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:52.367300 19724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:52.367710 19724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:52.369151 19724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:52.371025 19724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:52.371145 19724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:52.371281 19724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:52.371356 19724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:52.417204 19724 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:33801
I20260806 19:35:52.417358 19852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:33801 every 8 connection(s)
I20260806 19:35:52.419207 19724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:35:52.422360 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 19724
I20260806 19:35:52.423146 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:37507
--local_ip_for_outbound_sockets=127.10.25.3
--tserver_master_addrs=127.10.25.62:44085
--webserver_port=35463
--webserver_interface=127.10.25.3
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.25.3
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:35:52 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:52 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 19:35:52.448130 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:52.423622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:45519 (local address 127.10.25.62:44085)
0806 19:35:52.424442 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:52.424453 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:52.426369 (+ 1916us) server_negotiation.cc:407] Connection header received
0806 19:35:52.429440 (+ 3071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:52.429451 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:52.429659 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:52.429851 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:52.432741 (+ 2890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:52.433545 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:52.436147 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:52.436876 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:52.437068 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:52.447779 (+ 10711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:45519: BlockingRecv error: recv got EOF from 127.10.25.2:45519 (error 108)
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":400,"threads_started":1}
I20260806 19:35:52.450317 19856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:52.424599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:45519)
0806 19:35:52.426101 (+ 1502us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:52.426170 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:52.428837 (+ 2667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:52.430345 (+ 1508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:52.430359 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:52.431450 (+ 1091us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:52.432400 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:52.432433 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:52.433796 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:52.433801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:52.435883 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:52.435907 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:52.437579 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:52.437590 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:52.438010 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:52.438982 (+ 972us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:52.439028 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:52.447450 (+ 8422us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1066,"thread_start_us":772,"threads_started":1}
W20260806 19:35:52.451723 19853 heartbeater.cc:644] Failed to heartbeat to 127.10.25.62:44085 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.25.62:44085: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Aug 06 19:35:52 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:52 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:52.468205 19856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:52.452834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:42055)
0806 19:35:52.454108 (+ 1274us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:52.454137 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:52.454369 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:52.454954 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:52.454963 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:52.455607 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:52.456134 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:52.456159 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:52.458626 (+ 2467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:52.458633 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:52.459806 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:52.459825 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:52.461227 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:52.461236 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:52.461587 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:52.462515 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:52.462551 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:52.467847 (+ 5296us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280}
W20260806 19:35:52.470819 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:52.453787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:42055 (local address 127.10.25.62:44085)
0806 19:35:52.454077 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:52.454084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:52.454259 (+ 175us) server_negotiation.cc:407] Connection header received
0806 19:35:52.454547 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:52.454554 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:52.454643 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:52.454923 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:52.456773 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:52.458370 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:52.460030 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:52.460977 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:52.461178 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:52.470403 (+ 9225us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:42055: BlockingRecv error: recv got EOF from 127.10.25.2:42055 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 06 19:35:52 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:52 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 19:35:52.486897 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:52.469778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:41225 (local address 127.10.25.62:44085)
0806 19:35:52.471390 (+ 1612us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:52.471420 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:52.471459 (+ 39us) server_negotiation.cc:407] Connection header received
0806 19:35:52.471598 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:52.471608 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:52.471738 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:52.472187 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:52.473612 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:52.475046 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:52.477772 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:52.478856 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:52.479019 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:52.486529 (+ 7510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:41225: BlockingRecv error: recv got EOF from 127.10.25.2:41225 (error 108)
Metrics: {"server-negotiator.queue_time_us":1355,"thread_start_us":3077,"threads_started":1}
I20260806 19:35:52.487579 19856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:52.469617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:41225)
0806 19:35:52.469941 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:52.469964 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:52.470112 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:52.472212 (+ 2100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:52.472219 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:52.472738 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:52.473300 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:52.473324 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:52.475325 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:52.475333 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:52.477460 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:52.477517 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:52.479378 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:52.479385 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:52.479594 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:52.480445 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:52.480519 (+ 74us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:52.487242 (+ 6723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177}
W20260806 19:35:52.487972 19853 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 19:35:52.867367 19857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:35:52.867703 19857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:35:52.867808 19857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:35:52.880056 19857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:35:52.880211 19857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:35:52.880337 19857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:35:52.892699 19857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/krb5kdc/oryx_127.10.25.3.keytab
--principal=oryx/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:37507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=35463
--tserver_master_addrs=127.10.25.62:44085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:35:52.894238 19857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:35:52.895709 19857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:35:52.912935 19863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:52.912968 19866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:35:52.913772 19864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:35:52.913867 19857 server_base.cc:1034] running on GCE node
I20260806 19:35:52.914808 19857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:35:52.916316 19857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:35:52.917760 19857 hybrid_clock.cc:648] HybridClock initialized: now 1786044952917672 us; error 78 us; skew 500 ppm
I20260806 19:35:52.918407 19857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:35:52 dist-test-slave-ss4n krb5kdc[18966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044952, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 19:35:52 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:52.925143 19857 init.cc:377] Logged in from keytab as oryx/127.10.25.3@KRBTEST.COM (short username oryx)
Aug 06 19:35:52 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:52.927520 19857 webserver.cc:460] Webserver started at http://127.10.25.3:35463/ using document root <none> and password file <none>
I20260806 19:35:52.928436 19868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:52.912685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:47503)
0806 19:35:52.914166 (+ 1481us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:52.914187 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:52.914349 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:52.914736 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:52.914741 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:52.915393 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:52.915657 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:52.915673 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:52.917652 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:52.917664 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:52.919616 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:52.919644 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:52.921090 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:52.921109 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:52.921432 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:52.922650 (+ 1218us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:52.922688 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:52.927971 (+ 5283us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":1309,"threads_started":1}
W20260806 19:35:52.928746 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:52.913202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:47503 (local address 127.10.25.62:44085)
0806 19:35:52.913423 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:52.913429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:52.914293 (+ 864us) server_negotiation.cc:407] Connection header received
0806 19:35:52.914497 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:52.914501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:52.914575 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:52.914669 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:52.915893 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:52.917286 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:52.919901 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:52.920669 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:52.920800 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:52.928399 (+ 7599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:47503: BlockingRecv error: recv got EOF from 127.10.25.1:47503 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 19:35:52.929149 19857 fs_manager.cc:359] Metadata directory not provided
I20260806 19:35:52.929456 19857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:35:52.938203 19857 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.005s
I20260806 19:35:52.943532 19874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:52.945917 19857 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:35:52.946205 19857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "2d3d73453b024d5a8f7629600f02c0e9"
format_stamp: "Formatted at 2026-08-06 19:35:49 on dist-test-slave-ss4n"
server_key: "e325bac356caa48a7ea0c8777a432a5b"
server_key_iv: "0ff7a91b84b787696d7614a2b112c665"
server_key_version: "encryptionkey@0"
I20260806 19:35:52.947746 19857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:35:52.967036 19857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:35:52.975421 19857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:35:52.976163 19857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:35:52.977669 19857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:35:52.979547 19857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:35:52.979696 19857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:52.979843 19857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:35:52.979941 19857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:35:53.027765 19857 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:37507
I20260806 19:35:53.027832 19988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:37507 every 8 connection(s)
I20260806 19:35:53.029677 19857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:35:53.040409 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 19857
Aug 06 19:35:53 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:53 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 19:35:53.059919 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:53.033441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:48925 (local address 127.10.25.62:44085)
0806 19:35:53.033760 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:53.033766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:53.035844 (+ 2078us) server_negotiation.cc:407] Connection header received
0806 19:35:53.039581 (+ 3737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:53.039588 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:53.039776 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:53.040311 (+ 535us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:53.043817 (+ 3506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.045083 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.048100 (+ 3017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.049137 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.049298 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:53.059664 (+ 10366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:48925: BlockingRecv error: recv got EOF from 127.10.25.3:48925 (error 108)
Metrics: {"server-negotiator.queue_time_us":172}
I20260806 19:35:53.060576 19991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:53.033850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:48925)
0806 19:35:53.035457 (+ 1607us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:53.035540 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:53.038862 (+ 3322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:53.041087 (+ 2225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:53.041102 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:53.042325 (+ 1223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:53.043511 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:53.043552 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.045323 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.045329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:53.047841 (+ 2512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:53.047860 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.049783 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.049795 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:53.050480 (+ 685us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:53.051828 (+ 1348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:53.051867 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:53.058995 (+ 7128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1172,"thread_start_us":812,"threads_started":1}
W20260806 19:35:53.061671 19989 heartbeater.cc:644] Failed to heartbeat to 127.10.25.62:44085 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.25.62:44085: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
I20260806 19:35:53.065608 19993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:53.041573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.10.25.62:44085)
0806 19:35:53.042997 (+ 1424us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:53.043003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:53.043026 (+ 23us) server_negotiation.cc:407] Connection header received
0806 19:35:53.043089 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:53.043094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:53.043201 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:53.043341 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:53.045905 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.047337 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.050015 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.050879 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.051107 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:53.056291 (+ 5184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:35:53.056310 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:35:53.056342 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:35:53.056416 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:35:53.061626 (+ 5210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:53.062706 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:53.062715 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:53.062725 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:53.062988 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:35:53.063428 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:35:53.063433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:35:53.063438 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:35:53.063921 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:35:53.064321 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0806 19:35:53.064799 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 19:35:53.065251 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1261,"thread_start_us":1130,"threads_started":1}
Aug 06 19:35:53 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:53 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:53.072245 19991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:53.062453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:52765)
0806 19:35:53.062740 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:53.062763 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:53.062901 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:53.063351 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:53.063356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:53.063745 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:53.063907 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:53.063917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.064967 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.064972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:53.066259 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:53.066275 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.067209 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.067215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:53.067414 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:53.068625 (+ 1211us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:53.068645 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:53.071965 (+ 3320us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 19:35:53.072664 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:53.062570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:52765 (local address 127.10.25.62:44085)
0806 19:35:53.062878 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:53.062884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:53.062903 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:35:53.063063 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:53.063069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:53.063201 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:53.063316 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:53.064062 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.064811 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.066417 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.067046 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.067198 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:53.072326 (+ 5128us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:52765: BlockingRecv error: recv got EOF from 127.10.25.3:52765 (error 108)
Metrics: {"server-negotiator.queue_time_us":161}
Aug 06 19:35:53 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:53 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:53.084370 19991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:53.073015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:38795)
0806 19:35:53.073275 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:53.073297 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:53.073422 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:53.074090 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:53.074094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:53.074409 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:53.074586 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:53.074595 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.075854 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.075859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:53.076922 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:53.076937 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.078184 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.078189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:53.078378 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:53.079255 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:53.079283 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:53.084105 (+ 4822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145}
W20260806 19:35:53.084805 19989 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 19:35:53.085222 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:53.073224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:38795 (local address 127.10.25.62:44085)
0806 19:35:53.073589 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:53.073595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:53.073624 (+ 29us) server_negotiation.cc:407] Connection header received
0806 19:35:53.073729 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:53.073735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:53.073838 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:53.073976 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:53.074842 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.075643 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.077227 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.077993 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.078125 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:53.084920 (+ 6795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:38795: BlockingRecv error: recv got EOF from 127.10.25.3:38795 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 19:35:53 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:53 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:53.498737 19994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:53.488778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:55667)
0806 19:35:53.489359 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:53.489400 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:53.489622 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:53.490000 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:53.490005 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:53.490548 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:53.490745 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:53.490753 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.491904 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.491910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:53.493014 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:53.493027 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.493889 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.493896 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:53.494073 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:53.494750 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:53.494771 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:53.498333 (+ 3562us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":326,"threads_started":1}
W20260806 19:35:53.498929 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:53.488971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:55667 (local address 127.10.25.62:44085)
0806 19:35:53.489246 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:53.489250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:53.489527 (+ 277us) server_negotiation.cc:407] Connection header received
0806 19:35:53.489784 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:53.489789 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:53.489860 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:53.489989 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:53.490915 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.491735 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.493163 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.493704 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.493848 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:53.498632 (+ 4784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:55667: BlockingRecv error: recv got EOF from 127.10.25.2:55667 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 06 19:35:53 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:53 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:53.942415 19995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:53.930067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:48137)
0806 19:35:53.930733 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:53.930766 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:53.930914 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:53.931326 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:53.931330 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:53.931802 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:53.931989 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:53.932000 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.933958 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.933964 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:53.935335 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:53.935358 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.936229 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.936236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:53.936430 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:53.937221 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:53.937242 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:53.941442 (+ 4200us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":372,"threads_started":1}
W20260806 19:35:53.943240 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:53.930471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:48137 (local address 127.10.25.62:44085)
0806 19:35:53.930923 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:53.930928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:53.930954 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:35:53.931020 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:53.931026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:53.931132 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:53.931234 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:53.932318 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.933718 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.935537 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:53.936016 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:53.936184 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:53.942136 (+ 5952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:48137: BlockingRecv error: recv got EOF from 127.10.25.1:48137 (error 108)
Metrics: {"server-negotiator.queue_time_us":138}
Aug 06 19:35:54 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:54 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:54.097289 19996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:54.085676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:60047)
0806 19:35:54.086291 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:54.086309 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:54.086441 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:54.086798 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:54.086802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:54.087344 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:54.087563 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:54.087574 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:54.088849 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:54.088853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:54.090208 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:54.090227 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:54.091341 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:54.091348 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:54.091525 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:54.092310 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:54.092333 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:54.096999 (+ 4666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":343,"threads_started":1}
W20260806 19:35:54.097759 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:54.085837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:60047 (local address 127.10.25.62:44085)
0806 19:35:54.086079 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:54.086084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:54.086426 (+ 342us) server_negotiation.cc:407] Connection header received
0806 19:35:54.086560 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:54.086564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:54.086639 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:54.086745 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:54.087791 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:54.088665 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:54.090428 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:54.090991 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:54.091195 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:54.097296 (+ 6101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:60047: BlockingRecv error: recv got EOF from 127.10.25.3:60047 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 06 19:35:54 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:54 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:54.509696 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:54.499926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:45723)
0806 19:35:54.500493 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:54.500510 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:54.500631 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:54.500933 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:54.500936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:54.501351 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:54.501588 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:54.501599 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:54.503011 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:54.503018 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:54.503977 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:54.503991 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:54.504824 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:54.504829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:54.504946 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:54.505688 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:54.505707 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:54.509235 (+ 3528us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":367,"threads_started":1}
W20260806 19:35:54.509907 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:54.499988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:45723 (local address 127.10.25.62:44085)
0806 19:35:54.500268 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:54.500278 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:54.500594 (+ 316us) server_negotiation.cc:407] Connection header received
0806 19:35:54.500742 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:54.500746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:54.500810 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:54.500894 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:54.501723 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:54.502840 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:54.504168 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:54.504659 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:54.504778 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:54.509609 (+ 4831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:45723: BlockingRecv error: recv got EOF from 127.10.25.2:45723 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 06 19:35:54 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:54 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:54.955003 19998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:54.944222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:35109)
0806 19:35:54.944886 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:54.944905 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:54.945042 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:54.945573 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:54.945579 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:54.945979 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:54.946198 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:54.946209 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:54.947386 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:54.947391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:54.948412 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:54.948427 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:54.949434 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:54.949439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:54.949671 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:54.950512 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:54.950533 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:54.954570 (+ 4037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":348,"threads_started":1}
W20260806 19:35:54.955145 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:54.944508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:35109 (local address 127.10.25.62:44085)
0806 19:35:54.944721 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:54.944726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:54.945028 (+ 302us) server_negotiation.cc:407] Connection header received
0806 19:35:54.945182 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:54.945188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:54.945298 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:54.945411 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:54.946433 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:54.947216 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:54.948595 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:54.949194 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:54.949377 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:54.954837 (+ 5460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:35109: BlockingRecv error: recv got EOF from 127.10.25.1:35109 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 06 19:35:55 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:55 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:55.109651 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:55.098517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:60519)
0806 19:35:55.099160 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:55.099179 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:55.099323 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:55.099919 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:55.099923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:55.100372 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:55.100549 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:55.100558 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:55.102094 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:55.102100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:55.103225 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:55.103246 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:55.104343 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:55.104350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:55.104511 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:55.105238 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:55.105257 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:55.109230 (+ 3973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":380,"threads_started":1}
W20260806 19:35:55.109767 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:55.098603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:60519 (local address 127.10.25.62:44085)
0806 19:35:55.098789 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:55.098793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:55.099279 (+ 486us) server_negotiation.cc:407] Connection header received
0806 19:35:55.099591 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:55.099597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:55.099713 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:55.099854 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:55.100774 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:55.101803 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:55.103580 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:55.104118 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:55.104275 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:55.109459 (+ 5184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:60519: BlockingRecv error: recv got EOF from 127.10.25.3:60519 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 19:35:55 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:55 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:55.522615 20000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:55.510913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:56427)
0806 19:35:55.511669 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:55.511686 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:55.511868 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:55.512245 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:55.512280 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:55.512766 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:55.512982 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:55.512993 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:55.514346 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:55.514352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:55.515289 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:55.515300 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:55.516687 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:55.516694 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:55.516896 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:55.517658 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:55.517678 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:55.522296 (+ 4618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":482,"threads_started":1}
W20260806 19:35:55.523136 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:55.511173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:56427 (local address 127.10.25.62:44085)
0806 19:35:55.511436 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:55.511445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:55.511788 (+ 343us) server_negotiation.cc:407] Connection header received
0806 19:35:55.512029 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:55.512034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:55.512103 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:55.512216 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:55.513138 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:55.514194 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:55.515566 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:55.516394 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:55.516653 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:55.522628 (+ 5975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:56427: BlockingRecv error: recv got EOF from 127.10.25.2:56427 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 06 19:35:55 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:55 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:55.967965 20001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:55.956571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:39777)
0806 19:35:55.957173 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:55.957192 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:55.957331 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:55.957872 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:55.957876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:55.958284 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:55.958445 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:55.958454 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:55.959747 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:55.959752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:55.960866 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:55.960880 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:55.962175 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:55.962182 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:55.962361 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:55.963224 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:55.963243 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:55.967562 (+ 4319us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":344,"threads_started":1}
W20260806 19:35:55.968067 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:55.956769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:39777 (local address 127.10.25.62:44085)
0806 19:35:55.957036 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:55.957041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:55.957348 (+ 307us) server_negotiation.cc:407] Connection header received
0806 19:35:55.957444 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:55.957448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:55.957708 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:55.957819 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:55.958682 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:55.959528 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:55.961106 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:55.961932 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:55.962090 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:55.967789 (+ 5699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:39777: BlockingRecv error: recv got EOF from 127.10.25.1:39777 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 06 19:35:56 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:56 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:56.123412 20002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:56.111151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:58183)
0806 19:35:56.114065 (+ 2914us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:56.114085 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:56.114224 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:56.114718 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:56.114722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:56.115099 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:56.115286 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:56.115295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:56.116489 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:56.116496 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:56.117559 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:56.117574 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:56.118435 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:56.118441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:56.118662 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:56.119432 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:56.119451 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:56.122967 (+ 3516us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2767,"thread_start_us":2566,"threads_started":1}
W20260806 19:35:56.123553 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:56.111255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:58183 (local address 127.10.25.62:44085)
0806 19:35:56.111458 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:56.111465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:56.114212 (+ 2747us) server_negotiation.cc:407] Connection header received
0806 19:35:56.114388 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:56.114394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:56.114501 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:56.114664 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:56.115493 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:56.116297 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:56.117750 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:56.118244 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:56.118401 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:56.123242 (+ 4841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:58183: BlockingRecv error: recv got EOF from 127.10.25.3:58183 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 19:35:56 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:56 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:56.542924 20003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:56.523923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:34303)
0806 19:35:56.524712 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:56.524731 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:56.524932 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:56.525436 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:56.525442 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:56.526030 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:56.526192 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:56.526201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:56.527314 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:56.527318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:56.528339 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:56.528351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:56.529150 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:56.529155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:56.529325 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:56.537567 (+ 8242us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:56.537633 (+ 66us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:56.542589 (+ 4956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":339,"threads_started":1}
W20260806 19:35:56.543277 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:56.524019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:34303 (local address 127.10.25.62:44085)
0806 19:35:56.524273 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:56.524283 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:56.524827 (+ 544us) server_negotiation.cc:407] Connection header received
0806 19:35:56.525148 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:56.525153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:56.525238 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:56.525363 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:56.526331 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:56.527160 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:56.528496 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:56.528935 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:56.529037 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:56.542982 (+ 13945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:34303: BlockingRecv error: recv got EOF from 127.10.25.2:34303 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 06 19:35:56 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:56 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:56.980304 20004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:56.969454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:37251)
0806 19:35:56.970101 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:56.970119 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:56.970265 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:56.970775 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:56.970780 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:56.971322 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:56.971588 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:56.971609 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:56.973253 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:56.973259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:56.974413 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:56.974427 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:56.975257 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:56.975264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:56.975463 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:56.976241 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:56.976261 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:56.980004 (+ 3743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":342,"threads_started":1}
W20260806 19:35:56.981149 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:56.969709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:37251 (local address 127.10.25.62:44085)
0806 19:35:56.969966 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:56.969973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:56.970212 (+ 239us) server_negotiation.cc:407] Connection header received
0806 19:35:56.970410 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:56.970416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:56.970530 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:56.970670 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:56.971758 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:56.973053 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:56.974597 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:56.975074 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:56.975239 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:56.980537 (+ 5298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:37251: BlockingRecv error: recv got EOF from 127.10.25.1:37251 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 06 19:35:57 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:57 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:57.136395 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:57.124820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:37291)
0806 19:35:57.125625 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:57.125644 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:57.125833 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:57.126233 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:57.126239 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:57.126698 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:57.126864 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:57.126883 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:57.128064 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:57.128071 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:57.129290 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:57.129307 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:57.130208 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:57.130214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:57.130449 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:57.131575 (+ 1126us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:57.131604 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:57.136124 (+ 4520us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":459,"threads_started":1}
W20260806 19:35:57.136749 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:57.124968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:37291 (local address 127.10.25.62:44085)
0806 19:35:57.125386 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:57.125395 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:57.125750 (+ 355us) server_negotiation.cc:407] Connection header received
0806 19:35:57.125999 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:57.126004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:57.126077 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:57.126205 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:57.127055 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:57.127894 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:57.129504 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:57.130013 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:57.130178 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:57.136424 (+ 6246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:37291: BlockingRecv error: recv got EOF from 127.10.25.3:37291 (error 108)
Metrics: {"server-negotiator.queue_time_us":191}
Aug 06 19:35:57 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:57 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:57.555361 20006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:57.544500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:55245)
0806 19:35:57.545140 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:57.545159 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:57.545296 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:57.545717 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:57.545721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:57.546170 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:57.546363 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:57.546373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:57.547952 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:57.547958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:57.549277 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:57.549297 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:57.550226 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:57.550231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:57.550375 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:57.551338 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:57.551361 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:57.555087 (+ 3726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":346,"threads_started":1}
W20260806 19:35:57.555629 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:57.544802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:55245 (local address 127.10.25.62:44085)
0806 19:35:57.545038 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:57.545044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:57.545248 (+ 204us) server_negotiation.cc:407] Connection header received
0806 19:35:57.545497 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:57.545502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:57.545567 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:57.545663 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:57.546678 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:57.547760 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:57.549504 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:57.550030 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:57.550159 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:57.555355 (+ 5196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:55245: BlockingRecv error: recv got EOF from 127.10.25.2:55245 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 06 19:35:57 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:57 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:57.992683 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:57.981798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:36327)
0806 19:35:57.982544 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:57.982565 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:57.982766 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:57.983247 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:57.983252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:57.983702 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:57.983922 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:57.983936 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:57.985380 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:57.985385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:57.986478 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:57.986492 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:57.987306 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:57.987312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:57.987460 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:57.988181 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:57.988202 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:57.992430 (+ 4228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":429,"threads_started":1}
W20260806 19:35:57.993062 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:57.981945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:36327 (local address 127.10.25.62:44085)
0806 19:35:57.982188 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:57.982196 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:57.982751 (+ 555us) server_negotiation.cc:407] Connection header received
0806 19:35:57.982936 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:57.982941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:57.983038 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:57.983170 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:57.984154 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:57.985203 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:57.986653 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:57.987113 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:57.987222 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:57.992749 (+ 5527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:36327: BlockingRecv error: recv got EOF from 127.10.25.1:36327 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 06 19:35:58 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:58 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:58.147220 20008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:58.137693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:51165)
0806 19:35:58.138288 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:58.138304 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:58.138427 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:58.138776 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:58.138780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:58.139193 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:58.139423 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:58.139436 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:58.140683 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:58.140688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:58.141962 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:58.141977 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:58.142779 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:58.142785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:58.142916 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:58.143678 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:58.143697 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:58.146943 (+ 3246us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":309,"threads_started":1}
W20260806 19:35:58.147634 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:58.137767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:51165 (local address 127.10.25.62:44085)
0806 19:35:58.137987 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:58.137992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:58.138399 (+ 407us) server_negotiation.cc:407] Connection header received
0806 19:35:58.138557 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:58.138562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:58.138630 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:58.138731 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:58.139605 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:58.140532 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:58.142129 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:58.142601 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:58.142833 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:58.147207 (+ 4374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:51165: BlockingRecv error: recv got EOF from 127.10.25.3:51165 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 06 19:35:58 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:58 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:58.569288 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:58.556819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:44805)
0806 19:35:58.557904 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:58.557926 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:58.558194 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:58.558634 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:58.558638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:58.559190 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:58.559562 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:58.559573 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:58.560971 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:58.560979 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:58.562299 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:58.562333 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:58.563380 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:58.563387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:58.563604 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:58.564859 (+ 1255us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:58.564888 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:58.569050 (+ 4162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":846,"thread_start_us":697,"threads_started":1}
W20260806 19:35:58.569692 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:58.557044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:44805 (local address 127.10.25.62:44085)
0806 19:35:58.557381 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:58.557391 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:58.558090 (+ 699us) server_negotiation.cc:407] Connection header received
0806 19:35:58.558384 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:58.558389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:58.558473 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:58.558596 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:58.559741 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:58.560778 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:58.562627 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:58.563161 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:58.563329 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:58.569311 (+ 5982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:44805: BlockingRecv error: recv got EOF from 127.10.25.2:44805 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
Aug 06 19:35:59 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:59 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:59.006268 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:58.994145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:47729)
0806 19:35:58.994991 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:58.995018 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:58.995191 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:58.995924 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:58.995934 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:58.996640 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:58.996923 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:58.996941 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:58.998626 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:58.998631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:59.000059 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:59.000093 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:59.001141 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:59.001148 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:59.001305 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:59.002094 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:59.002116 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:59.005765 (+ 3649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":428,"threads_started":1}
W20260806 19:35:59.006467 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:58.994310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:47729 (local address 127.10.25.62:44085)
0806 19:35:58.994652 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:58.994662 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:58.995204 (+ 542us) server_negotiation.cc:407] Connection header received
0806 19:35:58.995394 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:58.995403 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:58.995531 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:58.995674 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:58.997223 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:58.998367 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:59.000385 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:59.000918 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:59.001053 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:59.006113 (+ 5060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:47729: BlockingRecv error: recv got EOF from 127.10.25.1:47729 (error 108)
Metrics: {"server-negotiator.queue_time_us":158}
Aug 06 19:35:59 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:59 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:59.160143 20011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:59.148467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:55081)
0806 19:35:59.149226 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:59.149247 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:59.149537 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:59.150087 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:59.150092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:59.150543 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:59.150746 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:59.150756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:59.152087 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:59.152091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:59.153066 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:59.153080 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:59.154409 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:59.154417 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:59.154619 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:59.155520 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:59.155544 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:59.159750 (+ 4206us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":447,"threads_started":1}
W20260806 19:35:59.160307 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:59.148582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:55081 (local address 127.10.25.62:44085)
0806 19:35:59.148783 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:59.148787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:59.149436 (+ 649us) server_negotiation.cc:407] Connection header received
0806 19:35:59.149770 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:59.149776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:59.149919 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:59.150017 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:59.150937 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:59.151871 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:59.153427 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:59.154121 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:59.154298 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:59.160016 (+ 5718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:55081: BlockingRecv error: recv got EOF from 127.10.25.3:55081 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 19:35:59 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:35:59 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:35:59.582252 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:35:59.570672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:60361)
0806 19:35:59.571541 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0806 19:35:59.571566 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 19:35:59.571911 (+ 345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:35:59.572411 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:35:59.572422 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:35:59.572908 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:35:59.573150 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:59.573160 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:59.575117 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:59.575123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:59.576372 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:35:59.576388 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:35:59.577265 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:35:59.577271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:35:59.577415 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:59.578268 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:35:59.578291 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:35:59.581939 (+ 3648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":443,"threads_started":1}
W20260806 19:35:59.582773 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:35:59.570871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:60361 (local address 127.10.25.62:44085)
0806 19:35:59.571302 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0806 19:35:59.571310 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:35:59.571717 (+ 407us) server_negotiation.cc:407] Connection header received
0806 19:35:59.572113 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:35:59.572119 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:35:59.572199 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:35:59.572374 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:35:59.573343 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:59.574944 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:59.576582 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:35:59.577070 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:35:59.577189 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:35:59.582296 (+ 5107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:60361: BlockingRecv error: recv got EOF from 127.10.25.2:60361 (error 108)
Metrics: {"server-negotiator.queue_time_us":142}
Aug 06 19:36:00 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:00 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:00.018625 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:00.007458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:49319)
0806 19:36:00.008056 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:00.008075 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:00.008222 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:00.008608 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:00.008613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:00.009189 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:00.009372 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:00.009382 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:00.010969 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:00.010976 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:00.012157 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:00.012179 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:00.013401 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:00.013417 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:00.013768 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:00.014729 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:00.014765 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:00.018286 (+ 3521us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":290,"threads_started":1}
W20260806 19:36:00.019323 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:00.007703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:49319 (local address 127.10.25.62:44085)
0806 19:36:00.007995 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:00.008002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:00.008198 (+ 196us) server_negotiation.cc:407] Connection header received
0806 19:36:00.008330 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:00.008335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:00.008408 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:00.008507 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:00.009654 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:00.010670 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:00.012451 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:00.013015 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:00.013161 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:00.019040 (+ 5879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:49319: BlockingRecv error: recv got EOF from 127.10.25.1:49319 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
Aug 06 19:36:00 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:00 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:00.173614 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:00.161943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:39647)
0806 19:36:00.162833 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:00.162856 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:00.163120 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:00.163615 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:00.163619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:00.164080 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:00.164299 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:00.164309 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:00.166086 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:00.166094 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:00.167705 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:00.167737 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:00.168725 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:00.168732 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:00.168923 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:00.169727 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:00.169747 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:00.173198 (+ 3451us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":750,"thread_start_us":406,"threads_started":1}
W20260806 19:36:00.174008 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:00.162233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:39647 (local address 127.10.25.62:44085)
0806 19:36:00.162619 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:00.162627 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:00.163131 (+ 504us) server_negotiation.cc:407] Connection header received
0806 19:36:00.163291 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:00.163296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:00.163381 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:00.163503 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:00.164521 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:00.165772 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:00.167927 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:00.168526 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:00.168692 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:00.173721 (+ 5029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:39647: BlockingRecv error: recv got EOF from 127.10.25.3:39647 (error 108)
Metrics: {"server-negotiator.queue_time_us":163}
Aug 06 19:36:00 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:00 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:00.595933 20015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:00.583763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:41231)
0806 19:36:00.584795 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:00.584817 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:00.585068 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:00.585571 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:00.585576 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:00.586417 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:00.586697 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:00.586712 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:00.588282 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:00.588297 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:00.589544 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:00.589563 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:00.590481 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:00.590491 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:00.590686 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:00.591398 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:00.591417 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:00.595502 (+ 4085us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":873,"thread_start_us":661,"threads_started":1}
W20260806 19:36:00.596652 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:00.584062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:41231 (local address 127.10.25.62:44085)
0806 19:36:00.584465 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:00.584474 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:00.584967 (+ 493us) server_negotiation.cc:407] Connection header received
0806 19:36:00.585276 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:00.585281 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:00.585371 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:00.585525 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:00.586953 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:00.588080 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:00.589770 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:00.590238 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:00.590402 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:00.595927 (+ 5525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:41231: BlockingRecv error: recv got EOF from 127.10.25.2:41231 (error 108)
Metrics: {"server-negotiator.queue_time_us":157}
Aug 06 19:36:01 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:01 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:01.030681 20016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:01.020086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:57307)
0806 19:36:01.020976 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:01.021004 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:01.021164 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:01.021610 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:01.021614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:01.022070 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:01.022273 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:01.022283 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:01.023478 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:01.023487 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:01.024608 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:01.024621 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:01.025523 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:01.025531 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:01.025736 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:01.026488 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:01.026513 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:01.030414 (+ 3901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":603,"threads_started":1}
W20260806 19:36:01.030972 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:01.020469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:57307 (local address 127.10.25.62:44085)
0806 19:36:01.020793 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:01.020798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:01.021139 (+ 341us) server_negotiation.cc:407] Connection header received
0806 19:36:01.021293 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:01.021297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:01.021375 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:01.021592 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:01.022452 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:01.023288 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:01.024780 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:01.025294 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:01.025450 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:01.030703 (+ 5253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:57307: BlockingRecv error: recv got EOF from 127.10.25.1:57307 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 06 19:36:01 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:01 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:01.184433 20017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:01.174888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:57649)
0806 19:36:01.175513 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:01.175532 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:01.175689 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:01.176055 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:01.176060 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:01.176486 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:01.176681 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:01.176690 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:01.177770 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:01.177775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:01.178820 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:01.178834 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:01.179666 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:01.179672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:01.179815 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:01.180550 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:01.180571 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:01.184180 (+ 3609us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":322,"threads_started":1}
W20260806 19:36:01.184762 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:01.175005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:57649 (local address 127.10.25.62:44085)
0806 19:36:01.175323 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:01.175328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:01.175683 (+ 355us) server_negotiation.cc:407] Connection header received
0806 19:36:01.175804 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:01.175808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:01.175874 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:01.175963 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:01.176841 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:01.177610 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:01.178980 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:01.179446 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:01.179601 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:01.184456 (+ 4855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:57649: BlockingRecv error: recv got EOF from 127.10.25.3:57649 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
Aug 06 19:36:01 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:01 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:01.611336 20018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:01.597452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:49067)
0806 19:36:01.598119 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:01.598138 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:01.598268 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:01.598997 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:01.599002 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:01.599487 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:01.599738 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:01.599749 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:01.600975 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:01.600980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:01.602114 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:01.602129 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:01.603210 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:01.603218 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:01.603442 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:01.607303 (+ 3861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:01.607327 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:01.611001 (+ 3674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":388,"threads_started":1}
W20260806 19:36:01.611660 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:01.597637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:49067 (local address 127.10.25.62:44085)
0806 19:36:01.597836 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:01.597840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:01.598242 (+ 402us) server_negotiation.cc:407] Connection header received
0806 19:36:01.598591 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:01.598598 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:01.598708 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:01.598866 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:01.599972 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:01.600795 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:01.602323 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:01.603004 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:01.603125 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:01.611297 (+ 8172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:49067: BlockingRecv error: recv got EOF from 127.10.25.2:49067 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 19:36:02 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:02 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:02.042588 20020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:02.031949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:42945)
0806 19:36:02.032481 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:02.032498 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:02.032649 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:02.033094 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:02.033098 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:02.033616 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:02.033781 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:02.033791 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:02.035208 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:02.035213 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:02.036295 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:02.036310 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:02.037122 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:02.037128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:02.037271 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:02.038099 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:02.038119 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:02.042229 (+ 4110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":303,"threads_started":1}
W20260806 19:36:02.042824 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:02.032154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:42945 (local address 127.10.25.62:44085)
0806 19:36:02.032407 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:02.032414 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:02.032591 (+ 177us) server_negotiation.cc:407] Connection header received
0806 19:36:02.032839 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:02.032844 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:02.032916 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:02.033021 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:02.033993 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:02.035009 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:02.036458 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:02.036939 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:02.037053 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:02.042469 (+ 5416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:42945: BlockingRecv error: recv got EOF from 127.10.25.1:42945 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 06 19:36:02 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:02 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:02.196208 20021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:02.185769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:44773)
0806 19:36:02.186665 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:02.186687 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:02.186846 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:02.187327 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:02.187331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:02.187933 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:02.188151 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:02.188161 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:02.189409 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:02.189413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:02.190452 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:02.190466 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:02.191343 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:02.191348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:02.191476 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:02.192238 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:02.192255 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:02.195934 (+ 3679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":769,"thread_start_us":633,"threads_started":1}
W20260806 19:36:02.196542 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:02.186398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:44773 (local address 127.10.25.62:44085)
0806 19:36:02.186676 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:02.186682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:02.186846 (+ 164us) server_negotiation.cc:407] Connection header received
0806 19:36:02.187011 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:02.187017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:02.187141 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:02.187263 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:02.188405 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:02.189256 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:02.190655 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:02.191167 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:02.191292 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:02.196253 (+ 4961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:44773: BlockingRecv error: recv got EOF from 127.10.25.3:44773 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 06 19:36:02 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:02 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:02.625201 20022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:02.612643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:54309)
0806 19:36:02.613292 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:02.613310 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:02.613438 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:02.613894 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:02.613899 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:02.614317 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:02.614489 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:02.614498 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:02.615797 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:02.615802 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:02.617773 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:02.617813 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:02.619476 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:02.619487 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:02.619738 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:02.620826 (+ 1088us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:02.620852 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:02.624740 (+ 3888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":401,"threads_started":1}
W20260806 19:36:02.625331 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:02.612833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:54309 (local address 127.10.25.62:44085)
0806 19:36:02.613047 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:02.613052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:02.613408 (+ 356us) server_negotiation.cc:407] Connection header received
0806 19:36:02.613619 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:02.613632 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:02.613701 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:02.613807 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:02.614695 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:02.615612 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:02.618236 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:02.618967 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:02.619170 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:02.625060 (+ 5890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:54309: BlockingRecv error: recv got EOF from 127.10.25.2:54309 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 19:36:03 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:03 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:03.054904 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:03.043917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:37955)
0806 19:36:03.044459 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:03.044482 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:03.044629 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:03.044984 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:03.044987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:03.045413 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:03.045742 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:03.045759 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:03.047229 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:03.047236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:03.048597 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:03.048616 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:03.050113 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:03.050123 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:03.050322 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:03.051047 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:03.051067 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:03.054609 (+ 3542us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":469,"threads_started":1}
W20260806 19:36:03.055248 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:03.044272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:37955 (local address 127.10.25.62:44085)
0806 19:36:03.044529 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:03.044534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:03.044557 (+ 23us) server_negotiation.cc:407] Connection header received
0806 19:36:03.044764 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:03.044768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:03.044832 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:03.044927 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:03.046050 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:03.046970 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:03.048770 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:03.049858 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:03.050036 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:03.054928 (+ 4892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:37955: BlockingRecv error: recv got EOF from 127.10.25.1:37955 (error 108)
Metrics: {"server-negotiator.queue_time_us":139}
Aug 06 19:36:03 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:03 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:03.208114 20024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:03.197428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:58629)
0806 19:36:03.198076 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:03.198094 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:03.198242 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:03.198618 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:03.198621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:03.199038 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:03.199190 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:03.199198 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:03.200461 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:03.200469 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:03.201947 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:03.201966 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:03.202957 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:03.202962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:03.203111 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:03.203919 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:03.203947 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:03.207743 (+ 3796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":406,"threads_started":1}
W20260806 19:36:03.208241 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:03.197636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:58629 (local address 127.10.25.62:44085)
0806 19:36:03.198069 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:03.198073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:03.198182 (+ 109us) server_negotiation.cc:407] Connection header received
0806 19:36:03.198397 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:03.198402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:03.198476 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:03.198570 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:03.199360 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:03.200146 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:03.202156 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:03.202735 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:03.202861 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:03.207971 (+ 5110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:58629: BlockingRecv error: recv got EOF from 127.10.25.3:58629 (error 108)
Metrics: {"server-negotiator.queue_time_us":239}
Aug 06 19:36:03 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:03 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:03.637979 20025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:03.626386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:41771)
0806 19:36:03.627245 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:03.627263 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:03.627382 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:03.627849 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:03.627853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:03.628298 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:03.628473 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:03.628482 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:03.629996 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:03.630003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:03.631234 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:03.631249 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:03.632142 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:03.632147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:03.632323 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:03.633129 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:03.633150 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:03.637475 (+ 4325us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":618,"threads_started":1}
W20260806 19:36:03.638104 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:03.626641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:41771 (local address 127.10.25.62:44085)
0806 19:36:03.626891 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:03.626900 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:03.627363 (+ 463us) server_negotiation.cc:407] Connection header received
0806 19:36:03.627576 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:03.627580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:03.627666 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:03.627768 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:03.628754 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:03.629785 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:03.631427 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:03.631945 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:03.632074 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:03.637793 (+ 5719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:41771: BlockingRecv error: recv got EOF from 127.10.25.2:41771 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 19:36:04 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:04 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:04.065743 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:04.055989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:38451)
0806 19:36:04.056539 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:04.056561 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:04.056677 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:04.057029 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:04.057033 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:04.057452 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:04.057645 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:04.057654 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:04.058902 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:04.058906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:04.059945 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:04.059966 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:04.060839 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:04.060844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:04.060999 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:04.061838 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:04.061857 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:04.065343 (+ 3486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":325,"threads_started":1}
W20260806 19:36:04.065909 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:04.056128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:38451 (local address 127.10.25.62:44085)
0806 19:36:04.056307 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:04.056312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:04.056645 (+ 333us) server_negotiation.cc:407] Connection header received
0806 19:36:04.056784 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:04.056787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:04.056861 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:04.056952 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:04.057869 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:04.058727 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:04.060150 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:04.060656 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:04.060786 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:04.065629 (+ 4843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:38451: BlockingRecv error: recv got EOF from 127.10.25.1:38451 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 19:36:04 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:04 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:04.220319 20027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:04.209938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:53449)
0806 19:36:04.210524 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:04.210548 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:04.210797 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:04.211627 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:04.211633 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:04.212225 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:04.212448 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:04.212459 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:04.214112 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:04.214126 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:04.215199 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:04.215213 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:04.216099 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:04.216104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:04.216236 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:04.216930 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:04.216963 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:04.220064 (+ 3101us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":457,"threads_started":1}
W20260806 19:36:04.220938 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:04.210090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:53449 (local address 127.10.25.62:44085)
0806 19:36:04.210346 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:04.210353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:04.210650 (+ 297us) server_negotiation.cc:407] Connection header received
0806 19:36:04.211110 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:04.211116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:04.211333 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:04.211538 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:04.212760 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:04.213918 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:04.215403 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:04.215883 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:04.216000 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:04.220567 (+ 4567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:53449: BlockingRecv error: recv got EOF from 127.10.25.3:53449 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 06 19:36:04 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:04 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:04.650502 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:04.639655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:50569)
0806 19:36:04.640353 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:04.640383 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:04.640588 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:04.640991 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:04.640997 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:04.641510 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:04.641690 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:04.641700 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:04.642950 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:04.642957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:04.644086 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:04.644100 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:04.645087 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:04.645095 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:04.645309 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:04.646445 (+ 1136us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:04.646473 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:04.650229 (+ 3756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":432,"threads_started":1}
W20260806 19:36:04.650915 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:04.639782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:50569 (local address 127.10.25.62:44085)
0806 19:36:04.640100 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:04.640110 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:04.640509 (+ 399us) server_negotiation.cc:407] Connection header received
0806 19:36:04.640750 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:04.640754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:04.640823 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:04.640949 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:04.641860 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:04.642785 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:04.644272 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:04.644891 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:04.645046 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:04.650573 (+ 5527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:50569: BlockingRecv error: recv got EOF from 127.10.25.2:50569 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 06 19:36:05 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:05 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:05.078402 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:05.067516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:60057)
0806 19:36:05.068287 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:05.068305 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:05.068441 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:05.068861 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:05.068865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:05.069306 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:05.069528 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:05.069538 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:05.070732 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:05.070737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:05.071795 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:05.071808 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:05.073015 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:05.073022 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:05.073216 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:05.073913 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:05.073932 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:05.078111 (+ 4179us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":433,"threads_started":1}
W20260806 19:36:05.078730 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:05.067710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:60057 (local address 127.10.25.62:44085)
0806 19:36:05.068086 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:05.068092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:05.068408 (+ 316us) server_negotiation.cc:407] Connection header received
0806 19:36:05.068617 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:05.068622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:05.068701 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:05.068809 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:05.069687 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:05.070540 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:05.072064 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:05.072721 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:05.072973 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:05.078444 (+ 5471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:60057: BlockingRecv error: recv got EOF from 127.10.25.1:60057 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 06 19:36:05 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:05 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:05.233579 20030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:05.222026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:34181)
0806 19:36:05.222685 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:05.222704 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:05.222933 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:05.223661 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:05.223667 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:05.224301 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:05.224487 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:05.224497 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:05.226237 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:05.226245 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:05.227463 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:05.227484 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:05.228659 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:05.228665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:05.228841 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:05.229801 (+ 960us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:05.229826 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:05.233219 (+ 3393us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":402,"threads_started":1}
W20260806 19:36:05.234017 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:05.222216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:34181 (local address 127.10.25.62:44085)
0806 19:36:05.222442 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:05.222447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:05.222800 (+ 353us) server_negotiation.cc:407] Connection header received
0806 19:36:05.223214 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:05.223218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:05.223360 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:05.223484 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:05.224857 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:05.225973 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:05.227776 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:05.228399 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:05.228582 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:05.233617 (+ 5035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:34181: BlockingRecv error: recv got EOF from 127.10.25.3:34181 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 06 19:36:05 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:05 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:05.662828 20031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:05.651826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:38445)
0806 19:36:05.652549 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:05.652567 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:05.652766 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:05.653203 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:05.653209 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:05.653762 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:05.653950 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:05.653961 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:05.655106 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:05.655112 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:05.656177 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:05.656191 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:05.657116 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:05.657122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:05.657322 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:05.658213 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:05.658248 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:05.662545 (+ 4297us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":402,"threads_started":1}
W20260806 19:36:05.663166 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:05.652159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:38445 (local address 127.10.25.62:44085)
0806 19:36:05.652444 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:05.652452 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:05.652671 (+ 219us) server_negotiation.cc:407] Connection header received
0806 19:36:05.652927 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:05.652938 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:05.653033 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:05.653158 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:05.654120 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:05.654931 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:05.656362 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:05.656894 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:05.657052 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:05.662886 (+ 5834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:38445: BlockingRecv error: recv got EOF from 127.10.25.2:38445 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 06 19:36:06 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:06 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:06.090703 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:06.079789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:37607)
0806 19:36:06.080400 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:06.080415 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:06.080590 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:06.080972 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:06.080976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:06.081395 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:06.081619 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:06.081630 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:06.082839 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:06.082846 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:06.084070 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:06.084101 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:06.085198 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:06.085206 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:06.085559 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:06.086501 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:06.086522 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:06.090416 (+ 3894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":361,"threads_started":1}
W20260806 19:36:06.091114 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:06.079943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:37607 (local address 127.10.25.62:44085)
0806 19:36:06.080269 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:06.080276 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:06.080538 (+ 262us) server_negotiation.cc:407] Connection header received
0806 19:36:06.080751 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:06.080761 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:06.080825 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:06.080937 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:06.081780 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:06.082675 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:06.084371 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:06.084965 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:06.085152 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:06.090767 (+ 5615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:37607: BlockingRecv error: recv got EOF from 127.10.25.1:37607 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
Aug 06 19:36:06 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:06 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:06.246762 20033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:06.235107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:55087)
0806 19:36:06.236336 (+ 1229us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:06.236354 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:06.236552 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:06.236961 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:06.236966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:06.237422 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:06.237688 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:06.237699 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:06.238814 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:06.238821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:06.239972 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:06.239989 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:06.240910 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:06.240920 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:06.241111 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:06.242141 (+ 1030us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:06.242186 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:06.246471 (+ 4285us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1123,"thread_start_us":647,"threads_started":1}
W20260806 19:36:06.247100 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:06.235296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:55087 (local address 127.10.25.62:44085)
0806 19:36:06.235762 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:06.235768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:06.236472 (+ 704us) server_negotiation.cc:407] Connection header received
0806 19:36:06.236696 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:06.236704 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:06.236805 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:06.236926 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:06.237852 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:06.238653 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:06.240166 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:06.240671 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:06.240829 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:06.246787 (+ 5958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:55087: BlockingRecv error: recv got EOF from 127.10.25.3:55087 (error 108)
Metrics: {"server-negotiator.queue_time_us":179}
Aug 06 19:36:06 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:06 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:06.681589 20034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:06.664374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:48721)
0806 19:36:06.664947 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:06.664976 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:06.665123 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:06.665655 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:06.665659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:06.666202 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:06.666412 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:06.666422 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:06.668157 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:06.668166 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:06.669814 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:06.669836 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:06.670739 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:06.670745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:06.670914 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:06.677748 (+ 6834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:06.677776 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:06.681099 (+ 3323us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":310,"threads_started":1}
W20260806 19:36:06.681761 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:06.664661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:48721 (local address 127.10.25.62:44085)
0806 19:36:06.664979 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:06.664985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:06.665072 (+ 87us) server_negotiation.cc:407] Connection header received
0806 19:36:06.665339 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:06.665345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:06.665429 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:06.665582 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:06.666573 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:06.667866 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:06.670012 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:06.670539 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:06.670673 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:06.681376 (+ 10703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:48721: BlockingRecv error: recv got EOF from 127.10.25.2:48721 (error 108)
Metrics: {"server-negotiator.queue_time_us":155}
Aug 06 19:36:07 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:07 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.1@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:07.103603 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:07.092209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.1:41315)
0806 19:36:07.092853 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:07.092871 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:07.093004 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:07.093372 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:07.093375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:07.093943 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:07.094170 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:07.094180 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:07.096096 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:07.096103 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:07.097757 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:07.097776 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:07.098675 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:07.098682 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:07.098826 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:07.099637 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:07.099659 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:07.103176 (+ 3517us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":363,"threads_started":1}
W20260806 19:36:07.103766 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:07.092427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:41315 (local address 127.10.25.62:44085)
0806 19:36:07.092647 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:07.092652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:07.092993 (+ 341us) server_negotiation.cc:407] Connection header received
0806 19:36:07.093146 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:07.093151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:07.093220 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:07.093320 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:07.094541 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:07.095814 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:07.097951 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:07.098473 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:07.098595 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:07.103476 (+ 4881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.1:41315: BlockingRecv error: recv got EOF from 127.10.25.1:41315 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 19:36:07 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:07 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.3@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:07.258203 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:07.247892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.3:58449)
0806 19:36:07.248411 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:07.248429 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:07.248567 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:07.249277 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:07.249282 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:07.250187 (+ 905us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:07.250424 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:07.250437 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:07.251702 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:07.251707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:07.252761 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:07.252775 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:07.253657 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:07.253663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:07.253788 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:07.254502 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:07.254521 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:07.257910 (+ 3389us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":290,"threads_started":1}
W20260806 19:36:07.258846 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:07.248009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:58449 (local address 127.10.25.62:44085)
0806 19:36:07.248297 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:07.248304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:07.248603 (+ 299us) server_negotiation.cc:407] Connection header received
0806 19:36:07.248727 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:07.248733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:07.248995 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:07.249139 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:07.250616 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:07.251490 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:07.252910 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:07.253395 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:07.253578 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:07.258504 (+ 4926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.3:58449: BlockingRecv error: recv got EOF from 127.10.25.3:58449 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
Aug 06 19:36:07 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 19:36:07 dist-test-slave-ss4n krb5kdc[18966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.25.2@KRBTEST.COM for oryx/127.10.25.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 19:36:07.696871 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:07.683503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44085 (local address 127.10.25.2:57061)
0806 19:36:07.684208 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:07.684226 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:07.684409 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:07.684825 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:07.684830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:07.685309 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:07.685585 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:07.685598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:07.687313 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:07.687320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:07.688723 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:07.688758 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:07.690061 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:07.690068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:07.690294 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:07.691574 (+ 1280us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:07.691602 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:07.696274 (+ 4672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.25.62:44085: Server oryx/127.10.25.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":403,"threads_started":1}
W20260806 19:36:07.697026 19855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:07.683588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:57061 (local address 127.10.25.62:44085)
0806 19:36:07.683971 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:07.683982 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:07.684350 (+ 368us) server_negotiation.cc:407] Connection header received
0806 19:36:07.684589 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:07.684594 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:07.684670 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:07.684793 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:07.685784 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:07.687128 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:07.689047 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:07.689798 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:07.690039 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:07.696683 (+ 6644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.25.2:57061: BlockingRecv error: recv got EOF from 127.10.25.2:57061 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 19:36:08.041013 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 19567
I20260806 19:36:08.057457 20039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:08.042473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59548 (local address 127.10.25.1:36743)
0806 19:36:08.043084 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:08.043095 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:08.043122 (+ 27us) server_negotiation.cc:407] Connection header received
0806 19:36:08.043195 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:08.043201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:08.043436 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:08.043697 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:08.045870 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:08.047398 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:08.049379 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:08.050285 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:08.050448 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:08.053586 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:08.053597 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:08.053614 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:36:08.053676 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:08.056850 (+ 3174us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.25.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 19:36:08.057199 (+ 349us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59548: Request ticket server kudu/127.10.25.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":317,"threads_started":1}
W20260806 19:36:08.057636 20039 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59548: Request ticket server kudu/127.10.25.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 19:36:08.057982 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 19567: Not authorized: Client connection negotiation failed: client connection to 127.10.25.1:36743: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.25.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 19:36:08.058244 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 19567
I20260806 19:36:08.081310 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 19724
I20260806 19:36:08.097213 20040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:08.082884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33582 (local address 127.10.25.2:33801)
0806 19:36:08.083561 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:08.083570 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:08.083604 (+ 34us) server_negotiation.cc:407] Connection header received
0806 19:36:08.083672 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:08.083679 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:08.083915 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:08.084180 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:08.085862 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:08.087137 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:08.089098 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:08.089874 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:08.090069 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:08.093113 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:08.093126 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:08.093143 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:36:08.093190 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:08.096310 (+ 3120us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.25.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 19:36:08.096800 (+ 490us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33582: Request ticket server kudu/127.10.25.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":342,"threads_started":1}
W20260806 19:36:08.097375 20040 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33582: Request ticket server kudu/127.10.25.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 19:36:08.097385 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 19724: Not authorized: Client connection negotiation failed: client connection to 127.10.25.2:33801: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.25.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 19:36:08.097723 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 19724
I20260806 19:36:08.121569 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 19857
I20260806 19:36:08.138094 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:08.122807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38188 (local address 127.10.25.3:37507)
0806 19:36:08.123375 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:08.123384 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:08.123412 (+ 28us) server_negotiation.cc:407] Connection header received
0806 19:36:08.123478 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:08.123486 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:08.123756 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:08.123922 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:08.125820 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:08.127376 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:08.129127 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:08.129912 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:08.130053 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:08.133711 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:08.133735 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:08.133767 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:36:08.133901 (+ 134us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:08.137282 (+ 3381us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.25.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 19:36:08.137777 (+ 495us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38188: Request ticket server kudu/127.10.25.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":295,"threads_started":1}
W20260806 19:36:08.138232 20041 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38188: Request ticket server kudu/127.10.25.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 19:36:08.138989 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 19857: Not authorized: Client connection negotiation failed: client connection to 127.10.25.3:37507: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.25.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 19:36:08.139187 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 19857
I20260806 19:36:08.162420 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 19496
I20260806 19:36:08.164484 19560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:36:08.291958 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 19496
2026-08-06T19:36:08Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22505 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T19:36:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:36:08Z Disabled control of system clock
I20260806 19:36:08.391119 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:41449
--webserver_interface=127.10.25.62
--webserver_port=0
--builtin_ntp_servers=127.10.25.20:39153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:41449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 19:36:08.788985 20046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:36:08.789350 20046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:36:08.789495 20046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:36:08.798448 20046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:36:08.798604 20046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:36:08.798686 20046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:36:08.798785 20046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:36:08.798853 20046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:36:08.809048 20046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:41449
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:41449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:36:08.810627 20046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:36:08.812628 20046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:36:08.826136 20051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:08.826176 20052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:08.826092 20054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:36:08.827730 20046 server_base.cc:1034] running on GCE node
I20260806 19:36:08.829342 20046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:36:08.831781 20046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:36:08.833237 20046 hybrid_clock.cc:648] HybridClock initialized: now 1786044968833135 us; error 81 us; skew 500 ppm
I20260806 19:36:08.833986 20046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:36:08.837124 20046 webserver.cc:460] Webserver started at http://127.10.25.62:36511/ using document root <none> and password file <none>
I20260806 19:36:08.838165 20046 fs_manager.cc:359] Metadata directory not provided
I20260806 19:36:08.838356 20046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:36:08.838789 20046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:36:08.842715 20046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "0205168a3ae04e8087624469028b4e2b"
format_stamp: "Formatted at 2026-08-06 19:36:08 on dist-test-slave-ss4n"
server_key: "d8f0ce47122c44a07f408e8e432d9e1c"
server_key_iv: "3290e03b073bdfaec42657cb18d1ac4e"
server_key_version: "encryptionkey@0"
I20260806 19:36:08.843926 20046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "0205168a3ae04e8087624469028b4e2b"
format_stamp: "Formatted at 2026-08-06 19:36:08 on dist-test-slave-ss4n"
server_key: "d8f0ce47122c44a07f408e8e432d9e1c"
server_key_iv: "3290e03b073bdfaec42657cb18d1ac4e"
server_key_version: "encryptionkey@0"
I20260806 19:36:08.850991 20046 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260806 19:36:08.856163 20060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:08.858271 20046 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 19:36:08.858507 20046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "0205168a3ae04e8087624469028b4e2b"
format_stamp: "Formatted at 2026-08-06 19:36:08 on dist-test-slave-ss4n"
server_key: "d8f0ce47122c44a07f408e8e432d9e1c"
server_key_iv: "3290e03b073bdfaec42657cb18d1ac4e"
server_key_version: "encryptionkey@0"
I20260806 19:36:08.858784 20046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:36:08.887253 20046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:36:08.888895 20046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:36:08.889302 20046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:36:08.912685 20046 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:41449
I20260806 19:36:08.912729 20112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:41449 every 8 connection(s)
I20260806 19:36:08.914377 20046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:36:08.917197 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 20046
I20260806 19:36:08.917502 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:36:08.918496 10340 external_mini_cluster.cc:1428] Setting key f2dae46d38066e8a556aa4a46907b436
I20260806 19:36:08.919662 20113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:36:08.937184 20113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b: Bootstrap starting.
I20260806 19:36:08.938789 20116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:08.921458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39594 (local address 127.10.25.62:41449)
0806 19:36:08.922825 (+ 1367us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:08.922844 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:08.922905 (+ 61us) server_negotiation.cc:407] Connection header received
0806 19:36:08.925671 (+ 2766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:08.925714 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:08.926543 (+ 829us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:08.927135 (+ 592us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:08.927915 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:08.930124 (+ 2209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:08.931937 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:08.933144 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:08.933409 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:08.935425 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:08.935440 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:08.935467 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:36:08.936315 (+ 848us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:08.936434 (+ 119us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:36:08.936470 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:08.936619 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:08.936924 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 19:36:08.937349 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":705,"thread_start_us":420,"threads_started":1}
I20260806 19:36:08.941583 20113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b: Neither blocks nor log segments found. Creating new log.
I20260806 19:36:08.943578 20113 log.cc:824] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b: Log is configured to *not* fsync() on all Append() calls
I20260806 19:36:08.947510 20113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b: No bootstrap required, opened a new log
I20260806 19:36:08.956764 20113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0205168a3ae04e8087624469028b4e2b" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41449 } }
I20260806 19:36:08.957226 20113 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:36:08.957357 20113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0205168a3ae04e8087624469028b4e2b, State: Initialized, Role: FOLLOWER
I20260806 19:36:08.958072 20113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0205168a3ae04e8087624469028b4e2b" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41449 } }
I20260806 19:36:08.958317 20113 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:36:08.958491 20113 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:36:08.958693 20113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:36:08.961077 20113 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0205168a3ae04e8087624469028b4e2b" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41449 } }
I20260806 19:36:08.961815 20113 leader_election.cc:302] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0205168a3ae04e8087624469028b4e2b; no voters:
I20260806 19:36:08.962888 20113 leader_election.cc:288] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:36:08.963227 20118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:36:08.964897 20118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [term 1 LEADER]: Becoming Leader. State: Replica: 0205168a3ae04e8087624469028b4e2b, State: Running, Role: LEADER
I20260806 19:36:08.965601 20113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:36:08.965852 20118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0205168a3ae04e8087624469028b4e2b" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41449 } }
I20260806 19:36:08.969856 20119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0205168a3ae04e8087624469028b4e2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0205168a3ae04e8087624469028b4e2b" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41449 } } }
I20260806 19:36:08.970607 20119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [sys.catalog]: This master's current role is: LEADER
I20260806 19:36:08.969838 20120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0205168a3ae04e8087624469028b4e2b. Latest consensus state: current_term: 1 leader_uuid: "0205168a3ae04e8087624469028b4e2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0205168a3ae04e8087624469028b4e2b" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41449 } } }
I20260806 19:36:08.971356 20120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b [sys.catalog]: This master's current role is: LEADER
I20260806 19:36:08.974022 20128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:36:08.981011 20128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:36:08.992851 20128 catalog_manager.cc:1347] Generated new cluster ID: 68a0fbb9fd804aaa859d00fad4d09ac9
I20260806 19:36:08.993119 20128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:36:09.011785 20128 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:36:09.013314 20128 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:36:09.029083 20128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0205168a3ae04e8087624469028b4e2b: Generated new TSK 0
I20260806 19:36:09.030673 20128 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:36:09.048192 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:41449
--builtin_ntp_servers=127.10.25.20:39153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 19:36:09.471827 20138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:36:09.472353 20138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:36:09.472497 20138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:36:09.481535 20138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:36:09.481712 20138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:36:09.481848 20138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:36:09.492272 20138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:41449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:36:09.493984 20138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:36:09.496093 20138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:36:09.512214 20147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:09.512230 20144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:09.512140 20145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:36:09.513060 20138 server_base.cc:1034] running on GCE node
I20260806 19:36:09.513890 20138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:36:09.515003 20138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:36:09.516255 20138 hybrid_clock.cc:648] HybridClock initialized: now 1786044969516176 us; error 67 us; skew 500 ppm
I20260806 19:36:09.516947 20138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:36:09.521087 20138 webserver.cc:460] Webserver started at http://127.10.25.1:46773/ using document root <none> and password file <none>
I20260806 19:36:09.522321 20138 fs_manager.cc:359] Metadata directory not provided
I20260806 19:36:09.522442 20138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:36:09.522791 20138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:36:09.526386 20138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "19f7b239afef4c70b3b8c9bc8acf58f3"
format_stamp: "Formatted at 2026-08-06 19:36:09 on dist-test-slave-ss4n"
server_key: "d779ae1e6c3090e439a0c0a974dc174e"
server_key_iv: "40f6fd80dab835dae0afd8e359eb3df6"
server_key_version: "encryptionkey@0"
I20260806 19:36:09.527362 20138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "19f7b239afef4c70b3b8c9bc8acf58f3"
format_stamp: "Formatted at 2026-08-06 19:36:09 on dist-test-slave-ss4n"
server_key: "d779ae1e6c3090e439a0c0a974dc174e"
server_key_iv: "40f6fd80dab835dae0afd8e359eb3df6"
server_key_version: "encryptionkey@0"
I20260806 19:36:09.534024 20138 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260806 19:36:09.538707 20153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:09.540715 20138 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 19:36:09.541005 20138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "19f7b239afef4c70b3b8c9bc8acf58f3"
format_stamp: "Formatted at 2026-08-06 19:36:09 on dist-test-slave-ss4n"
server_key: "d779ae1e6c3090e439a0c0a974dc174e"
server_key_iv: "40f6fd80dab835dae0afd8e359eb3df6"
server_key_version: "encryptionkey@0"
I20260806 19:36:09.541286 20138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:36:09.577240 20138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:36:09.578287 20138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:36:09.578645 20138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:36:09.579872 20138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:36:09.581612 20138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:36:09.581710 20138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:09.581847 20138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:36:09.581910 20138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:09.626869 20138 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:44567
I20260806 19:36:09.627049 20267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:44567 every 8 connection(s)
I20260806 19:36:09.628468 20138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:36:09.635567 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 20138
I20260806 19:36:09.635843 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:36:09.636554 10340 external_mini_cluster.cc:1428] Setting key fd538434461abace138aea835ef63d64
I20260806 19:36:09.639918 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:41449
--builtin_ntp_servers=127.10.25.20:39153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 19:36:09.643144 20270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:09.631688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:43931 (local address 127.10.25.62:41449)
0806 19:36:09.632207 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:09.632212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:09.633448 (+ 1236us) server_negotiation.cc:407] Connection header received
0806 19:36:09.635446 (+ 1998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:09.635452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:09.635557 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:09.635744 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:09.637171 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:09.638213 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:09.640008 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:09.640492 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:09.640646 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:09.642248 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:09.642255 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:09.642258 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:36:09.642278 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:09.642319 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:36:09.642331 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:09.642436 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:09.642670 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 19:36:09.642775 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":295,"threads_started":1}
I20260806 19:36:09.644498 20271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:09.632202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:41449 (local address 127.10.25.1:43931)
0806 19:36:09.633262 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:09.633308 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:09.635118 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:09.636145 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:09.636154 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:09.636266 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:09.636966 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:09.636992 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:09.638398 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:09.638403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:09.639836 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:09.639850 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:09.640770 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:09.640780 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:09.641014 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:09.641865 (+ 851us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:36:09.641868 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:09.641912 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:36:09.642006 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:36:09.642435 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:36:09.642449 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:36:09.642460 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 19:36:09.642669 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 19:36:09.643120 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":370,"threads_started":1}
I20260806 19:36:09.646306 20268 heartbeater.cc:347] Connected to a master server at 127.10.25.62:41449
I20260806 19:36:09.646673 20268 heartbeater.cc:464] Registering TS with master...
I20260806 19:36:09.647675 20268 heartbeater.cc:511] Master 127.10.25.62:41449 requested a full tablet report, sending...
I20260806 19:36:09.650168 20077 ts_manager.cc:194] Registered new tserver with Master: 19f7b239afef4c70b3b8c9bc8acf58f3 (127.10.25.1:44567)
I20260806 19:36:09.652112 20077 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.25.1:43931
W20260806 19:36:10.037647 20272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:36:10.038040 20272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:36:10.038203 20272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:36:10.046187 20272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:36:10.046365 20272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:36:10.046555 20272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:36:10.056524 20272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:41449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:36:10.058097 20272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:36:10.060077 20272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:36:10.075878 20277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:10.075878 20278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:10.075892 20280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:36:10.077443 20272 server_base.cc:1034] running on GCE node
I20260806 19:36:10.078311 20272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:36:10.079505 20272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:36:10.080793 20272 hybrid_clock.cc:648] HybridClock initialized: now 1786044970080697 us; error 86 us; skew 500 ppm
I20260806 19:36:10.081236 20272 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:36:10.083931 20272 webserver.cc:460] Webserver started at http://127.10.25.2:46141/ using document root <none> and password file <none>
I20260806 19:36:10.084759 20272 fs_manager.cc:359] Metadata directory not provided
I20260806 19:36:10.084858 20272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:36:10.085222 20272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:36:10.088526 20272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "1c8474380d244a0ea1c15c153f3cb663"
format_stamp: "Formatted at 2026-08-06 19:36:10 on dist-test-slave-ss4n"
server_key: "63880eec972369e00bd53a733cd6c752"
server_key_iv: "cce407dced46a27020dbfb00ced3d190"
server_key_version: "encryptionkey@0"
I20260806 19:36:10.090087 20272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "1c8474380d244a0ea1c15c153f3cb663"
format_stamp: "Formatted at 2026-08-06 19:36:10 on dist-test-slave-ss4n"
server_key: "63880eec972369e00bd53a733cd6c752"
server_key_iv: "cce407dced46a27020dbfb00ced3d190"
server_key_version: "encryptionkey@0"
I20260806 19:36:10.098006 20272 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260806 19:36:10.102722 20286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:10.104979 20272 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:36:10.105260 20272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "1c8474380d244a0ea1c15c153f3cb663"
format_stamp: "Formatted at 2026-08-06 19:36:10 on dist-test-slave-ss4n"
server_key: "63880eec972369e00bd53a733cd6c752"
server_key_iv: "cce407dced46a27020dbfb00ced3d190"
server_key_version: "encryptionkey@0"
I20260806 19:36:10.105607 20272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:36:10.121887 20272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:36:10.123008 20272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:36:10.123292 20272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:36:10.124455 20272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:36:10.126197 20272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:36:10.126307 20272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:10.126407 20272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:36:10.126482 20272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:10.169662 20272 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:39919
I20260806 19:36:10.169806 20402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:39919 every 8 connection(s)
I20260806 19:36:10.171298 20272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:36:10.173995 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 20272
I20260806 19:36:10.174208 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:36:10.174835 10340 external_mini_cluster.cc:1428] Setting key 49a224c6bd0943ca21ff105916fced78
I20260806 19:36:10.178543 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:41449
--builtin_ntp_servers=127.10.25.20:39153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 19:36:10.187227 20405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:10.174482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:52511 (local address 127.10.25.62:41449)
0806 19:36:10.175027 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:10.175032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:10.176957 (+ 1925us) server_negotiation.cc:407] Connection header received
0806 19:36:10.179281 (+ 2324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:10.179288 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:10.179397 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:10.179547 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:10.180922 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:10.181854 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:10.183760 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:10.184252 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:10.184414 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:10.186193 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:10.186199 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:10.186202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:36:10.186226 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:10.186272 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:36:10.186286 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:10.186400 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:10.186677 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 19:36:10.186800 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":293,"threads_started":1}
I20260806 19:36:10.188576 20406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:10.175326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:41449 (local address 127.10.25.2:52511)
0806 19:36:10.176734 (+ 1408us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:10.176797 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:10.178950 (+ 2153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:10.179869 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:10.179880 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:10.180006 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:10.180715 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:10.180735 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:10.182056 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:10.182060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:10.183565 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:10.183581 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:10.184555 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:10.184565 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:10.184825 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:10.185774 (+ 949us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:36:10.185777 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:10.185825 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:36:10.185925 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:36:10.186402 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:36:10.186418 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:36:10.186429 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 19:36:10.186666 (+ 237us) client_negotiation.cc:240] Negotiation successful
0806 19:36:10.187178 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":494,"threads_started":1}
I20260806 19:36:10.190640 20403 heartbeater.cc:347] Connected to a master server at 127.10.25.62:41449
I20260806 19:36:10.191010 20403 heartbeater.cc:464] Registering TS with master...
I20260806 19:36:10.191933 20403 heartbeater.cc:511] Master 127.10.25.62:41449 requested a full tablet report, sending...
I20260806 19:36:10.194012 20077 ts_manager.cc:194] Registered new tserver with Master: 1c8474380d244a0ea1c15c153f3cb663 (127.10.25.2:39919)
I20260806 19:36:10.194922 20077 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.25.2:52511
W20260806 19:36:10.579010 20407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:36:10.579346 20407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:36:10.579442 20407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:36:10.587538 20407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:36:10.587695 20407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:36:10.587879 20407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:36:10.598753 20407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:41449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:36:10.600206 20407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:36:10.602325 20407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:36:10.617617 20415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:10.617588 20413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:10.617549 20412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:36:10.618472 20407 server_base.cc:1034] running on GCE node
I20260806 19:36:10.619292 20407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:36:10.620831 20407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:36:10.622146 20407 hybrid_clock.cc:648] HybridClock initialized: now 1786044970622039 us; error 70 us; skew 500 ppm
I20260806 19:36:10.622577 20407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:36:10.625784 20407 webserver.cc:460] Webserver started at http://127.10.25.3:46039/ using document root <none> and password file <none>
I20260806 19:36:10.626888 20407 fs_manager.cc:359] Metadata directory not provided
I20260806 19:36:10.627040 20407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:36:10.627482 20407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:36:10.631786 20407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "c218944dce984b02ad91f95d8e4b5412"
format_stamp: "Formatted at 2026-08-06 19:36:10 on dist-test-slave-ss4n"
server_key: "b3912a77490f33ab1881dc14d5174744"
server_key_iv: "baea014aba6f98f04b35cd7c9540c052"
server_key_version: "encryptionkey@0"
I20260806 19:36:10.632977 20407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "c218944dce984b02ad91f95d8e4b5412"
format_stamp: "Formatted at 2026-08-06 19:36:10 on dist-test-slave-ss4n"
server_key: "b3912a77490f33ab1881dc14d5174744"
server_key_iv: "baea014aba6f98f04b35cd7c9540c052"
server_key_version: "encryptionkey@0"
I20260806 19:36:10.639369 20407 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 19:36:10.643821 20421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:10.645709 20407 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:36:10.645936 20407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "c218944dce984b02ad91f95d8e4b5412"
format_stamp: "Formatted at 2026-08-06 19:36:10 on dist-test-slave-ss4n"
server_key: "b3912a77490f33ab1881dc14d5174744"
server_key_iv: "baea014aba6f98f04b35cd7c9540c052"
server_key_version: "encryptionkey@0"
I20260806 19:36:10.646209 20407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:36:10.655548 20268 heartbeater.cc:503] Master 127.10.25.62:41449 was elected leader, sending a full tablet report...
I20260806 19:36:10.665285 20407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:36:10.666894 20407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:36:10.667435 20407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:36:10.668883 20407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:36:10.670848 20407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:36:10.670985 20407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:10.671136 20407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:36:10.671229 20407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:10.709328 20407 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:43963
I20260806 19:36:10.709364 20537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:43963 every 8 connection(s)
I20260806 19:36:10.710951 20407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:36:10.717062 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 20407
I20260806 19:36:10.717267 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:36:10.717876 10340 external_mini_cluster.cc:1428] Setting key 99bb005d6325198132abf63eff3d6d6e
I20260806 19:36:10.727211 20540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:10.714686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:58509 (local address 127.10.25.62:41449)
0806 19:36:10.715424 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:10.715431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:10.717187 (+ 1756us) server_negotiation.cc:407] Connection header received
0806 19:36:10.719579 (+ 2392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:10.719587 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:10.719712 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:10.719860 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:10.721113 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:10.722048 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:10.723792 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:10.724227 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:10.724376 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:10.726093 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:10.726099 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:10.726104 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:36:10.726128 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:10.726186 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:36:10.726202 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:10.726325 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:10.726605 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 19:36:10.726732 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":305,"threads_started":1}
I20260806 19:36:10.728459 20541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:10.715719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:41449 (local address 127.10.25.3:58509)
0806 19:36:10.716948 (+ 1229us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:10.717003 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:10.719215 (+ 2212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:10.720157 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:10.720167 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:10.720280 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:10.720920 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:10.720938 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:10.722247 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:10.722252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:10.723600 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:10.723617 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:10.724494 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:10.724501 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:10.724767 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:10.725629 (+ 862us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:36:10.725632 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:10.725682 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:36:10.725780 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:36:10.726324 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:36:10.726338 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:36:10.726350 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 19:36:10.726592 (+ 242us) client_negotiation.cc:240] Negotiation successful
0806 19:36:10.727131 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":429,"threads_started":1}
I20260806 19:36:10.730512 20538 heartbeater.cc:347] Connected to a master server at 127.10.25.62:41449
I20260806 19:36:10.730877 20538 heartbeater.cc:464] Registering TS with master...
I20260806 19:36:10.731818 20538 heartbeater.cc:511] Master 127.10.25.62:41449 requested a full tablet report, sending...
I20260806 19:36:10.733764 20076 ts_manager.cc:194] Registered new tserver with Master: c218944dce984b02ad91f95d8e4b5412 (127.10.25.3:43963)
I20260806 19:36:10.734864 20076 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.25.3:58509
I20260806 19:36:10.735926 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 19:36:10.744858 20540 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:10.742315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39600 (local address 127.10.25.62:41449)
0806 19:36:10.742563 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:10.742568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:10.743139 (+ 571us) server_negotiation.cc:407] Connection header received
0806 19:36:10.743440 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:10.743447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:10.743536 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:10.743654 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:10.744528 (+ 874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39600: BlockingRecv error: recv got EOF from 127.0.0.1:39600 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 19:36:10.744935 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 20138
I20260806 19:36:10.755028 20548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:10.746230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43712 (local address 127.10.25.1:44567)
0806 19:36:10.746940 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:10.746947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:10.746977 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:36:10.747043 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:10.747049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:10.747317 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:10.747574 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:10.748103 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:10.749409 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:10.751024 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:10.751969 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:10.752153 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:10.753395 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:10.753404 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:10.753419 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:36:10.754131 (+ 712us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:10.754184 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:36:10.754206 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:10.754306 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:10.754573 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 19:36:10.754696 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":358,"threads_started":1}
I20260806 19:36:10.756495 20263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:36:10.860234 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 20138
I20260806 19:36:10.883183 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 20272
I20260806 19:36:10.893664 20550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:10.884241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35568 (local address 127.10.25.2:39919)
0806 19:36:10.884960 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:10.884968 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:10.885001 (+ 33us) server_negotiation.cc:407] Connection header received
0806 19:36:10.885077 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:10.885084 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:10.885314 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:10.885465 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:10.886004 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:10.887131 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:10.889028 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:10.889969 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:10.890138 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:10.891779 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:10.891788 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:10.891805 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:36:10.892498 (+ 693us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:10.892555 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:36:10.892587 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:10.892710 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:10.893030 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 19:36:10.893192 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":399,"threads_started":1}
I20260806 19:36:10.895107 20398 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:36:10.986795 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 20272
I20260806 19:36:11.010368 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 20407
I20260806 19:36:11.022038 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:11.011947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50672 (local address 127.10.25.3:43963)
0806 19:36:11.012745 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:11.012760 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:11.012806 (+ 46us) server_negotiation.cc:407] Connection header received
0806 19:36:11.012882 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:11.012890 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:11.013171 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:11.013367 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:11.014196 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:11.015362 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:11.017413 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:11.018287 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:11.018437 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:11.019952 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:11.019963 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:11.019979 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:36:11.020688 (+ 709us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:11.020751 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:36:11.020775 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:11.021133 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:11.021359 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 19:36:11.021615 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":436,"threads_started":1}
I20260806 19:36:11.023572 20533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:36:11.115502 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 20407
I20260806 19:36:11.138711 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 20046
I20260806 19:36:11.140563 20108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:36:11.257412 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 20046
2026-08-06T19:36:11Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2967 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T19:36:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:36:11Z Disabled control of system clock
I20260806 19:36:11.363477 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:37007
--webserver_interface=127.10.25.62
--webserver_port=0
--builtin_ntp_servers=127.10.25.20:43515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:37007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 19:36:11.796159 20558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:36:11.796521 20558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:36:11.796684 20558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:36:11.806952 20558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:36:11.807174 20558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:36:11.807318 20558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:36:11.807456 20558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:36:11.807543 20558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:36:11.822458 20558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:43515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:37007
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:37007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:36:11.824602 20558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:36:11.828085 20558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:36:11.849007 20566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:11.849248 20564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:11.849980 20563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:36:11.850284 20558 server_base.cc:1034] running on GCE node
I20260806 19:36:11.851292 20558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:36:11.853165 20558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:36:11.854488 20558 hybrid_clock.cc:648] HybridClock initialized: now 1786044971854418 us; error 73 us; skew 500 ppm
I20260806 19:36:11.854964 20558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:36:11.867672 20558 webserver.cc:460] Webserver started at http://127.10.25.62:40139/ using document root <none> and password file <none>
I20260806 19:36:11.868661 20558 fs_manager.cc:359] Metadata directory not provided
I20260806 19:36:11.868847 20558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:36:11.869259 20558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:36:11.876001 20558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "8536d30434b44d86ae21aa320a547d7a"
format_stamp: "Formatted at 2026-08-06 19:36:11 on dist-test-slave-ss4n"
server_key: "124dc0a10c67266c1fd3da6db171155b"
server_key_iv: "44793c555ce3c6047b19fe112f240180"
server_key_version: "encryptionkey@0"
I20260806 19:36:11.877585 20558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "8536d30434b44d86ae21aa320a547d7a"
format_stamp: "Formatted at 2026-08-06 19:36:11 on dist-test-slave-ss4n"
server_key: "124dc0a10c67266c1fd3da6db171155b"
server_key_iv: "44793c555ce3c6047b19fe112f240180"
server_key_version: "encryptionkey@0"
I20260806 19:36:11.884775 20558 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.004s
I20260806 19:36:11.890376 20572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:11.892784 20558 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260806 19:36:11.893062 20558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "8536d30434b44d86ae21aa320a547d7a"
format_stamp: "Formatted at 2026-08-06 19:36:11 on dist-test-slave-ss4n"
server_key: "124dc0a10c67266c1fd3da6db171155b"
server_key_iv: "44793c555ce3c6047b19fe112f240180"
server_key_version: "encryptionkey@0"
I20260806 19:36:11.893365 20558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:36:11.902412 20558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:36:11.903522 20558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:36:11.903923 20558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:36:11.936834 20624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:37007 every 8 connection(s)
I20260806 19:36:11.936884 20558 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:37007
I20260806 19:36:11.938936 20558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:36:11.944703 20625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:36:11.945124 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 20558
I20260806 19:36:11.945617 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:36:11.946413 10340 external_mini_cluster.cc:1428] Setting key 3867ea8b264d0c4635f9f0479b5b3f71
I20260806 19:36:11.959481 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:11.949691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47818 (local address 127.10.25.62:37007)
0806 19:36:11.951739 (+ 2048us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:11.951760 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:11.951838 (+ 78us) server_negotiation.cc:407] Connection header received
0806 19:36:11.954077 (+ 2239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:11.954128 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:11.954925 (+ 797us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:11.955546 (+ 621us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:11.957009 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:11.957019 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:11.957039 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:36:11.957061 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:11.957159 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:36:11.957186 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:11.957533 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:11.957691 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 19:36:11.958016 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1082,"thread_start_us":658,"threads_started":1}
I20260806 19:36:11.965025 20625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a: Bootstrap starting.
I20260806 19:36:11.968859 20625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a: Neither blocks nor log segments found. Creating new log.
I20260806 19:36:11.970367 20625 log.cc:824] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:36:11.974285 20625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a: No bootstrap required, opened a new log
I20260806 19:36:11.983239 20625 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8536d30434b44d86ae21aa320a547d7a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37007 } }
I20260806 19:36:11.983664 20625 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:36:11.983768 20625 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8536d30434b44d86ae21aa320a547d7a, State: Initialized, Role: FOLLOWER
I20260806 19:36:11.984402 20625 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8536d30434b44d86ae21aa320a547d7a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37007 } }
I20260806 19:36:11.984651 20625 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:36:11.984802 20625 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:36:11.984995 20625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:36:11.987589 20625 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8536d30434b44d86ae21aa320a547d7a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37007 } }
I20260806 19:36:11.988533 20625 leader_election.cc:302] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8536d30434b44d86ae21aa320a547d7a; no voters:
I20260806 19:36:11.990140 20625 leader_election.cc:288] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:36:11.990274 20630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:36:11.992033 20630 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [term 1 LEADER]: Becoming Leader. State: Replica: 8536d30434b44d86ae21aa320a547d7a, State: Running, Role: LEADER
I20260806 19:36:11.992775 20630 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8536d30434b44d86ae21aa320a547d7a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37007 } }
I20260806 19:36:11.994087 20625 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:36:11.996966 20632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8536d30434b44d86ae21aa320a547d7a. Latest consensus state: current_term: 1 leader_uuid: "8536d30434b44d86ae21aa320a547d7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8536d30434b44d86ae21aa320a547d7a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37007 } } }
I20260806 19:36:11.996986 20631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8536d30434b44d86ae21aa320a547d7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8536d30434b44d86ae21aa320a547d7a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37007 } } }
I20260806 19:36:11.998456 20632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [sys.catalog]: This master's current role is: LEADER
I20260806 19:36:11.998473 20631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a [sys.catalog]: This master's current role is: LEADER
I20260806 19:36:12.001065 20636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:36:12.007872 20636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:36:12.022358 20636 catalog_manager.cc:1347] Generated new cluster ID: 56f24da5c5fb4c9bad910656ce5b6fd6
I20260806 19:36:12.022529 20636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:36:12.046435 20636 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:36:12.047302 20636 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:36:12.060292 20636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8536d30434b44d86ae21aa320a547d7a: Generated new TSK 0
I20260806 19:36:12.062126 20636 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:36:12.082657 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:37007
--builtin_ntp_servers=127.10.25.20:43515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 19:36:12.510941 20649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:36:12.511318 20649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:36:12.511456 20649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:36:12.519423 20649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:36:12.519554 20649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:36:12.519682 20649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:36:12.531209 20649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:43515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:37007
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:36:12.533612 20649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:36:12.537034 20649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:36:12.554670 20655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:12.554936 20656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:12.554731 20658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:36:12.556847 20649 server_base.cc:1034] running on GCE node
I20260806 19:36:12.557843 20649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:36:12.559171 20649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:36:12.560631 20649 hybrid_clock.cc:648] HybridClock initialized: now 1786044972560497 us; error 80 us; skew 500 ppm
I20260806 19:36:12.561347 20649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:36:12.565275 20649 webserver.cc:460] Webserver started at http://127.10.25.1:42625/ using document root <none> and password file <none>
I20260806 19:36:12.566547 20649 fs_manager.cc:359] Metadata directory not provided
I20260806 19:36:12.566689 20649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:36:12.567153 20649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:36:12.573853 20649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "bd6f7ea1b5cd42e3904d818033471ad8"
format_stamp: "Formatted at 2026-08-06 19:36:12 on dist-test-slave-ss4n"
server_key: "6443a2df670fdf60836ef8d45a48b592"
server_key_iv: "a7be7f50c59e248335aba6233372365d"
server_key_version: "encryptionkey@0"
I20260806 19:36:12.575488 20649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "bd6f7ea1b5cd42e3904d818033471ad8"
format_stamp: "Formatted at 2026-08-06 19:36:12 on dist-test-slave-ss4n"
server_key: "6443a2df670fdf60836ef8d45a48b592"
server_key_iv: "a7be7f50c59e248335aba6233372365d"
server_key_version: "encryptionkey@0"
I20260806 19:36:12.584636 20649 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.008s sys 0.000s
I20260806 19:36:12.590619 20665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:12.593042 20649 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.002s
I20260806 19:36:12.593319 20649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "bd6f7ea1b5cd42e3904d818033471ad8"
format_stamp: "Formatted at 2026-08-06 19:36:12 on dist-test-slave-ss4n"
server_key: "6443a2df670fdf60836ef8d45a48b592"
server_key_iv: "a7be7f50c59e248335aba6233372365d"
server_key_version: "encryptionkey@0"
I20260806 19:36:12.593870 20649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:36:12.606895 20649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:36:12.608397 20649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:36:12.608891 20649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:36:12.610849 20649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:36:12.613147 20649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:36:12.613296 20649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:12.613430 20649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:36:12.613538 20649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 19:36:12.662977 20649 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:33393
I20260806 19:36:12.663043 20780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:33393 every 8 connection(s)
I20260806 19:36:12.664739 20649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:36:12.674806 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 20649
I20260806 19:36:12.675119 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:36:12.675871 10340 external_mini_cluster.cc:1428] Setting key 4e6988f54d25f54aa944d2fe70629fb8
I20260806 19:36:12.676892 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:12.668576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:48835 (local address 127.10.25.62:37007)
0806 19:36:12.669324 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:12.669330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:12.671607 (+ 2277us) server_negotiation.cc:407] Connection header received
0806 19:36:12.673747 (+ 2140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:12.673754 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:12.673848 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:12.673981 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:12.675643 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:12.675650 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:12.675655 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:36:12.675688 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:12.675738 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:36:12.675750 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:12.675904 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:12.676256 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 19:36:12.676372 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":418,"threads_started":1}
I20260806 19:36:12.677966 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:12.669605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:37007 (local address 127.10.25.1:48835)
0806 19:36:12.671351 (+ 1746us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:12.671408 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:12.673314 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:12.674286 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:12.674296 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:12.674408 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:12.675286 (+ 878us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:36:12.675289 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:12.675334 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:36:12.675438 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:36:12.675856 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:36:12.675867 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:36:12.675877 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 19:36:12.676219 (+ 342us) client_negotiation.cc:240] Negotiation successful
0806 19:36:12.676473 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":890,"thread_start_us":557,"threads_started":1}
I20260806 19:36:12.680254 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:37007
--builtin_ntp_servers=127.10.25.20:43515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 19:36:12.680328 20781 heartbeater.cc:347] Connected to a master server at 127.10.25.62:37007
I20260806 19:36:12.680796 20781 heartbeater.cc:464] Registering TS with master...
I20260806 19:36:12.681643 20781 heartbeater.cc:511] Master 127.10.25.62:37007 requested a full tablet report, sending...
I20260806 19:36:12.685573 20589 ts_manager.cc:194] Registered new tserver with Master: bd6f7ea1b5cd42e3904d818033471ad8 (127.10.25.1:33393)
W20260806 19:36:13.093266 20786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:36:13.093714 20786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:36:13.093890 20786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:36:13.102213 20786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:36:13.102387 20786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:36:13.102577 20786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:36:13.112483 20786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:43515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:37007
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:36:13.114027 20786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:36:13.116079 20786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:36:13.131778 20795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:13.131803 20792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:13.131819 20791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:36:13.133186 20786 server_base.cc:1034] running on GCE node
I20260806 19:36:13.134060 20786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:36:13.135324 20786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:36:13.136656 20786 hybrid_clock.cc:648] HybridClock initialized: now 1786044973136605 us; error 44 us; skew 500 ppm
I20260806 19:36:13.137136 20786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:36:13.140280 20786 webserver.cc:460] Webserver started at http://127.10.25.2:34205/ using document root <none> and password file <none>
I20260806 19:36:13.141206 20786 fs_manager.cc:359] Metadata directory not provided
I20260806 19:36:13.141386 20786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:36:13.141829 20786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:36:13.146061 20786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "6a3f023418954506aab77e549eacff19"
format_stamp: "Formatted at 2026-08-06 19:36:13 on dist-test-slave-ss4n"
server_key: "583f9303602d86cc8794d626c0e2bdcf"
server_key_iv: "f5c7e1a1e083799c899fd662ce433028"
server_key_version: "encryptionkey@0"
I20260806 19:36:13.147877 20786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "6a3f023418954506aab77e549eacff19"
format_stamp: "Formatted at 2026-08-06 19:36:13 on dist-test-slave-ss4n"
server_key: "583f9303602d86cc8794d626c0e2bdcf"
server_key_iv: "f5c7e1a1e083799c899fd662ce433028"
server_key_version: "encryptionkey@0"
I20260806 19:36:13.156185 20786 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260806 19:36:13.161796 20802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:13.164218 20786 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 19:36:13.164448 20786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "6a3f023418954506aab77e549eacff19"
format_stamp: "Formatted at 2026-08-06 19:36:13 on dist-test-slave-ss4n"
server_key: "583f9303602d86cc8794d626c0e2bdcf"
server_key_iv: "f5c7e1a1e083799c899fd662ce433028"
server_key_version: "encryptionkey@0"
I20260806 19:36:13.164710 20786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:36:13.172377 20786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:36:13.173434 20786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:36:13.173827 20786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:36:13.174988 20786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:36:13.176813 20786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:36:13.176927 20786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:13.177016 20786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:36:13.177095 20786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:13.221314 20786 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:41407
I20260806 19:36:13.221326 20916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:41407 every 8 connection(s)
I20260806 19:36:13.223796 20786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:36:13.228124 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 20786
I20260806 19:36:13.228411 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:36:13.228941 10340 external_mini_cluster.cc:1428] Setting key 7215b9294a07ace6adbefc0ceac897e5
I20260806 19:36:13.232390 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:37007
--builtin_ntp_servers=127.10.25.20:43515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 19:36:13.233932 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:13.227088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:42435 (local address 127.10.25.62:37007)
0806 19:36:13.227639 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:13.227644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:13.229191 (+ 1547us) server_negotiation.cc:407] Connection header received
0806 19:36:13.231118 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:13.231124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:13.231201 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:13.231327 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:13.233008 (+ 1681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:13.233015 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:13.233018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:36:13.233041 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:13.233088 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:36:13.233102 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:13.233265 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:13.233506 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 19:36:13.233602 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":278,"threads_started":1}
I20260806 19:36:13.236119 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:13.227751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:37007 (local address 127.10.25.2:42435)
0806 19:36:13.228987 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:13.229038 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:13.230786 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:13.231596 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:13.231605 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:13.231719 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:13.232547 (+ 828us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:36:13.232551 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:13.232658 (+ 107us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:36:13.232790 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:36:13.233219 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:36:13.233233 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:36:13.233244 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 19:36:13.233620 (+ 376us) client_negotiation.cc:240] Negotiation successful
0806 19:36:13.233947 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":399,"threads_started":1}
I20260806 19:36:13.238101 20917 heartbeater.cc:347] Connected to a master server at 127.10.25.62:37007
I20260806 19:36:13.238422 20917 heartbeater.cc:464] Registering TS with master...
I20260806 19:36:13.239108 20917 heartbeater.cc:511] Master 127.10.25.62:37007 requested a full tablet report, sending...
I20260806 19:36:13.240893 20589 ts_manager.cc:194] Registered new tserver with Master: 6a3f023418954506aab77e549eacff19 (127.10.25.2:41407)
W20260806 19:36:13.671449 20921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:36:13.671824 20921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:36:13.672001 20921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:36:13.680306 20921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:36:13.680514 20921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:36:13.680708 20921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:36:13.690228 20781 heartbeater.cc:503] Master 127.10.25.62:37007 was elected leader, sending a full tablet report...
I20260806 19:36:13.692240 20921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:43515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:37007
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:36:13.694033 20921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:36:13.696835 20921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:36:13.716578 20926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:13.716564 20927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:13.716653 20929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:36:13.717857 20921 server_base.cc:1034] running on GCE node
I20260806 19:36:13.718672 20921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:36:13.719920 20921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:36:13.721225 20921 hybrid_clock.cc:648] HybridClock initialized: now 1786044973721180 us; error 63 us; skew 500 ppm
I20260806 19:36:13.722033 20921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:36:13.726006 20921 webserver.cc:460] Webserver started at http://127.10.25.3:42439/ using document root <none> and password file <none>
I20260806 19:36:13.726948 20921 fs_manager.cc:359] Metadata directory not provided
I20260806 19:36:13.727062 20921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:36:13.727451 20921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:36:13.731068 20921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "57f47cba3a8b40ea8e94510ca1e9b906"
format_stamp: "Formatted at 2026-08-06 19:36:13 on dist-test-slave-ss4n"
server_key: "9ce1242baf9e612efef91386e1afef56"
server_key_iv: "2ac33476d790580ef0978aaed0ab6860"
server_key_version: "encryptionkey@0"
I20260806 19:36:13.732084 20921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "57f47cba3a8b40ea8e94510ca1e9b906"
format_stamp: "Formatted at 2026-08-06 19:36:13 on dist-test-slave-ss4n"
server_key: "9ce1242baf9e612efef91386e1afef56"
server_key_iv: "2ac33476d790580ef0978aaed0ab6860"
server_key_version: "encryptionkey@0"
I20260806 19:36:13.739921 20921 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260806 19:36:13.747192 20935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:13.749320 20921 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:36:13.749558 20921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "57f47cba3a8b40ea8e94510ca1e9b906"
format_stamp: "Formatted at 2026-08-06 19:36:13 on dist-test-slave-ss4n"
server_key: "9ce1242baf9e612efef91386e1afef56"
server_key_iv: "2ac33476d790580ef0978aaed0ab6860"
server_key_version: "encryptionkey@0"
I20260806 19:36:13.749873 20921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:36:13.756808 20921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:36:13.758677 20921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:36:13.759182 20921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:36:13.760663 20921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:36:13.763080 20921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:36:13.763216 20921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:13.763329 20921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:36:13.763406 20921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:13.810639 20921 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:41617
I20260806 19:36:13.810791 21050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:41617 every 8 connection(s)
I20260806 19:36:13.812661 20921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:36:13.821266 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 20921
I20260806 19:36:13.821575 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:36:13.822062 10340 external_mini_cluster.cc:1428] Setting key b6cb0e0185b44b04d4d339accb85c57c
I20260806 19:36:13.825425 21053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:13.816587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:52813 (local address 127.10.25.62:37007)
0806 19:36:13.817333 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:13.817339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:13.819505 (+ 2166us) server_negotiation.cc:407] Connection header received
0806 19:36:13.821931 (+ 2426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:13.821938 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:13.822037 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:13.822197 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:13.824501 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:13.824515 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:13.824518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:36:13.824538 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:13.824585 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:36:13.824598 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:13.824736 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:13.824968 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 19:36:13.825044 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":458,"threads_started":1}
I20260806 19:36:13.826568 21054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:13.817709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:37007 (local address 127.10.25.3:52813)
0806 19:36:13.819269 (+ 1560us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:13.819332 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:13.821419 (+ 2087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:13.822504 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:13.822515 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:13.822642 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:13.824031 (+ 1389us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:36:13.824035 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:13.824098 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:36:13.824285 (+ 187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:36:13.824706 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:36:13.824718 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:36:13.824727 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 19:36:13.825061 (+ 334us) client_negotiation.cc:240] Negotiation successful
0806 19:36:13.825394 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":382,"threads_started":1}
I20260806 19:36:13.828702 21051 heartbeater.cc:347] Connected to a master server at 127.10.25.62:37007
I20260806 19:36:13.829085 21051 heartbeater.cc:464] Registering TS with master...
I20260806 19:36:13.829856 21051 heartbeater.cc:511] Master 127.10.25.62:37007 requested a full tablet report, sending...
I20260806 19:36:13.831686 20589 ts_manager.cc:194] Registered new tserver with Master: 57f47cba3a8b40ea8e94510ca1e9b906 (127.10.25.3:41617)
I20260806 19:36:13.840217 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 19:36:13.848682 21053 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:36:13.846587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47828 (local address 127.10.25.62:37007)
0806 19:36:13.846811 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:13.846815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:13.847399 (+ 584us) server_negotiation.cc:407] Connection header received
0806 19:36:13.847537 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:13.847544 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:13.847633 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:13.847852 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:13.848460 (+ 608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47828: BlockingRecv error: recv got EOF from 127.0.0.1:47828 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 19:36:13.850363 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 20649
I20260806 19:36:13.856455 21063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:13.852249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55088 (local address 127.10.25.1:33393)
0806 19:36:13.852979 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:13.852986 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:13.853023 (+ 37us) server_negotiation.cc:407] Connection header received
0806 19:36:13.853100 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:13.853107 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:13.853344 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:13.853600 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:13.855088 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:13.855097 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:13.855111 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:36:13.855131 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:13.855195 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:36:13.855214 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:13.855445 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:13.855514 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 19:36:13.855702 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":348,"threads_started":1}
I20260806 19:36:13.859021 20776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:36:13.973907 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 20649
I20260806 19:36:13.999930 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 20786
I20260806 19:36:14.006933 21065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:14.002310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40658 (local address 127.10.25.2:41407)
0806 19:36:14.003306 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:14.003318 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:14.003370 (+ 52us) server_negotiation.cc:407] Connection header received
0806 19:36:14.003490 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:14.003499 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:14.003962 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:14.004202 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:14.005872 (+ 1670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:14.005881 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:14.005895 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:36:14.005913 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:14.006065 (+ 152us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:36:14.006102 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:14.006254 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:14.006381 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 19:36:14.006542 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":380,"threads_started":1}
I20260806 19:36:14.008872 20912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:36:14.118858 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 20786
I20260806 19:36:14.146013 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 20921
I20260806 19:36:14.151762 21067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:14.147512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55278 (local address 127.10.25.3:41617)
0806 19:36:14.148637 (+ 1125us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:14.148645 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:14.148683 (+ 38us) server_negotiation.cc:407] Connection header received
0806 19:36:14.148760 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:14.148767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:14.149048 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:14.149208 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:14.150875 (+ 1667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:14.150883 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:14.150896 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:36:14.150922 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:14.150982 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:36:14.151001 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:14.151181 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:14.151359 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 19:36:14.151459 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":874,"thread_start_us":656,"threads_started":1}
I20260806 19:36:14.154074 21046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:36:14.254209 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 20921
I20260806 19:36:14.275985 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 20558
I20260806 19:36:14.277869 20620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:36:14.393388 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 20558
2026-08-06T19:36:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (3131 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:36:14 dist-test-slave-ss4n krb5kdc[21072](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:36:14 dist-test-slave-ss4n krb5kdc[21072](info): set up 2 sockets
Aug 06 19:36:14 dist-test-slave-ss4n krb5kdc[21072](info): commencing operation
krb5kdc: starting...
W20260806 19:36:15.651826 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:36:15 dist-test-slave-ss4n krb5kdc[21072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:36:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:36:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:36:16.070291 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:42489
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:36217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:42489
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:36:16.599058 21085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:36:16.599684 21085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:36:16.599970 21085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:36:16.609906 21085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:36:16.610069 21085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:36:16.610122 21085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:36:16.610177 21085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:36:16.610215 21085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:36:16.622908 21085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:36217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:42489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:42489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:36:16.624660 21085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:36:16.626202 21085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:36:16.648651 21090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:16.648701 21091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:16.648701 21093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:36:16.651437 21085 server_base.cc:1034] running on GCE node
I20260806 19:36:16.652590 21085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:36:16.655249 21085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:36:16.656672 21085 hybrid_clock.cc:648] HybridClock initialized: now 1786044976656561 us; error 96 us; skew 500 ppm
I20260806 19:36:16.657194 21085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:36:16 dist-test-slave-ss4n krb5kdc[21072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044976, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:36:16.662060 21085 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:36:16.665021 21085 webserver.cc:460] Webserver started at http://127.10.25.62:40069/ using document root <none> and password file <none>
I20260806 19:36:16.666507 21085 fs_manager.cc:359] Metadata directory not provided
I20260806 19:36:16.666863 21085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:36:16.667482 21085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:36:16.672369 21085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "274a377a5dca4cd8a6a33a103ed44068"
format_stamp: "Formatted at 2026-08-06 19:36:16 on dist-test-slave-ss4n"
server_key: "da0f310543e3bb228bfe19ca8acd3a36"
server_key_iv: "e129f1ae20e9df4a0a35b4b46cf06e8e"
server_key_version: "encryptionkey@0"
I20260806 19:36:16.674299 21085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "274a377a5dca4cd8a6a33a103ed44068"
format_stamp: "Formatted at 2026-08-06 19:36:16 on dist-test-slave-ss4n"
server_key: "da0f310543e3bb228bfe19ca8acd3a36"
server_key_iv: "e129f1ae20e9df4a0a35b4b46cf06e8e"
server_key_version: "encryptionkey@0"
I20260806 19:36:16.683979 21085 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.006s sys 0.001s
I20260806 19:36:16.692763 21100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:16.695742 21085 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260806 19:36:16.695984 21085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "274a377a5dca4cd8a6a33a103ed44068"
format_stamp: "Formatted at 2026-08-06 19:36:16 on dist-test-slave-ss4n"
server_key: "da0f310543e3bb228bfe19ca8acd3a36"
server_key_iv: "e129f1ae20e9df4a0a35b4b46cf06e8e"
server_key_version: "encryptionkey@0"
I20260806 19:36:16.696219 21085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:36:16.725833 21085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:36:16.732286 21085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:36:16.732623 21085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:36:16.761967 21152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:42489 every 8 connection(s)
I20260806 19:36:16.761967 21085 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:42489
I20260806 19:36:16.764318 21085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:36:16.765403 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 21085
I20260806 19:36:16.765719 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:36:16.766721 10340 external_mini_cluster.cc:1428] Setting key f0251b2f69c99108a1d433e0a0e7101c
I20260806 19:36:16.772440 21153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 19:36:16 dist-test-slave-ss4n krb5kdc[21072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:36:16.792810 21153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068: Bootstrap starting.
I20260806 19:36:16.797505 21153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068: Neither blocks nor log segments found. Creating new log.
I20260806 19:36:16.798988 21153 log.cc:824] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068: Log is configured to *not* fsync() on all Append() calls
I20260806 19:36:16.803689 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:16.769566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40030 (local address 127.10.25.62:42489)
0806 19:36:16.770507 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:16.770525 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:16.770581 (+ 56us) server_negotiation.cc:407] Connection header received
0806 19:36:16.773779 (+ 3198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:16.773814 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:16.774411 (+ 597us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:16.774979 (+ 568us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:16.776846 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:16.779172 (+ 2326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:16.780963 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:16.784347 (+ 3384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:16.784644 (+ 297us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:16.791778 (+ 7134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:16.791792 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:16.791820 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:36:16.791890 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:16.796214 (+ 4324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:36:16.797832 (+ 1618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:36:16.797845 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:36:16.797858 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:36:16.798137 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:36:16.798667 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:36:16.798673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:36:16.798681 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:36:16.799435 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:16.799958 (+ 523us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:16.800665 (+ 707us) server_negotiation.cc:299] Negotiation successful
0806 19:36:16.801419 (+ 754us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":367,"threads_started":1}
I20260806 19:36:16.803946 21153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068: No bootstrap required, opened a new log
I20260806 19:36:16.814553 21153 raft_consensus.cc:348] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "274a377a5dca4cd8a6a33a103ed44068" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 42489 } }
I20260806 19:36:16.815097 21153 raft_consensus.cc:374] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:36:16.815238 21153 raft_consensus.cc:729] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 274a377a5dca4cd8a6a33a103ed44068, State: Initialized, Role: FOLLOWER
I20260806 19:36:16.816054 21153 consensus_queue.cc:260] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "274a377a5dca4cd8a6a33a103ed44068" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 42489 } }
I20260806 19:36:16.816304 21153 raft_consensus.cc:388] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:36:16.816468 21153 raft_consensus.cc:482] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:36:16.816687 21153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:36:16.820168 21153 raft_consensus.cc:504] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "274a377a5dca4cd8a6a33a103ed44068" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 42489 } }
I20260806 19:36:16.820894 21153 leader_election.cc:302] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 274a377a5dca4cd8a6a33a103ed44068; no voters:
I20260806 19:36:16.821959 21153 leader_election.cc:288] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:36:16.822124 21158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:36:16.823029 21158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [term 1 LEADER]: Becoming Leader. State: Replica: 274a377a5dca4cd8a6a33a103ed44068, State: Running, Role: LEADER
I20260806 19:36:16.824689 21153 sys_catalog.cc:564] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:36:16.824231 21158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "274a377a5dca4cd8a6a33a103ed44068" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 42489 } }
I20260806 19:36:16.831933 21160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 274a377a5dca4cd8a6a33a103ed44068. Latest consensus state: current_term: 1 leader_uuid: "274a377a5dca4cd8a6a33a103ed44068" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "274a377a5dca4cd8a6a33a103ed44068" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 42489 } } }
I20260806 19:36:16.832060 21159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "274a377a5dca4cd8a6a33a103ed44068" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "274a377a5dca4cd8a6a33a103ed44068" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 42489 } } }
I20260806 19:36:16.832594 21159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [sys.catalog]: This master's current role is: LEADER
I20260806 19:36:16.832589 21160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068 [sys.catalog]: This master's current role is: LEADER
I20260806 19:36:16.835379 21168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:36:16.842595 21168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:36:16.857183 21168 catalog_manager.cc:1347] Generated new cluster ID: 684802a0470a435daf8a5e8db908bcfb
I20260806 19:36:16.857360 21168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:36:16.875676 21168 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:36:16.877437 21168 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:36:16.890408 21168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068: Generated new TSK 0
I20260806 19:36:16.891566 21168 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:36:16.995067 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:42489
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:36217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:36:17.446558 21181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:36:17.446895 21181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:36:17.447000 21181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:36:17.457196 21181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:36:17.457420 21181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:36:17.457778 21181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:36:17.471180 21181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:36217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:42489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:36:17.472859 21181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:36:17.474427 21181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:36:17.491025 21186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:17.491025 21187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:17.491014 21189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:36:17.493242 21181 server_base.cc:1034] running on GCE node
I20260806 19:36:17.494392 21181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:36:17.496062 21181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:36:17.497633 21181 hybrid_clock.cc:648] HybridClock initialized: now 1786044977497482 us; error 54 us; skew 500 ppm
I20260806 19:36:17.498066 21181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:36:17 dist-test-slave-ss4n krb5kdc[21072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044977, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:36:17.506289 21181 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:36:17.508083 21181 webserver.cc:460] Webserver started at http://127.10.25.1:39929/ using document root <none> and password file <none>
I20260806 19:36:17.508878 21181 fs_manager.cc:359] Metadata directory not provided
I20260806 19:36:17.509008 21181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:36:17.509296 21181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:36:17.512728 21181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "b9999987864444b8a1acee2b645f70ca"
format_stamp: "Formatted at 2026-08-06 19:36:17 on dist-test-slave-ss4n"
server_key: "267f44048321dcc63e0f80bc2ab5f9f9"
server_key_iv: "b51d07ad3dcbd2de6f69eef218b8086e"
server_key_version: "encryptionkey@0"
I20260806 19:36:17.513756 21181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "b9999987864444b8a1acee2b645f70ca"
format_stamp: "Formatted at 2026-08-06 19:36:17 on dist-test-slave-ss4n"
server_key: "267f44048321dcc63e0f80bc2ab5f9f9"
server_key_iv: "b51d07ad3dcbd2de6f69eef218b8086e"
server_key_version: "encryptionkey@0"
I20260806 19:36:17.522648 21181 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260806 19:36:17.527805 21196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:17.529613 21181 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 19:36:17.529794 21181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "b9999987864444b8a1acee2b645f70ca"
format_stamp: "Formatted at 2026-08-06 19:36:17 on dist-test-slave-ss4n"
server_key: "267f44048321dcc63e0f80bc2ab5f9f9"
server_key_iv: "b51d07ad3dcbd2de6f69eef218b8086e"
server_key_version: "encryptionkey@0"
I20260806 19:36:17.530000 21181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:36:17.553176 21181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:36:17.559629 21181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:36:17.559926 21181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:36:17.561131 21181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:36:17.563122 21181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:36:17.563216 21181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:17.563318 21181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:36:17.563355 21181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:17.606621 21181 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:40663
I20260806 19:36:17.606729 21313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:40663 every 8 connection(s)
I20260806 19:36:17.608022 21181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:36:17.617161 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 21181
I20260806 19:36:17.617707 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:36:17.618494 10340 external_mini_cluster.cc:1428] Setting key 0c556e2ea90bf6ec1425aa96009fd3d3
Aug 06 19:36:17 dist-test-slave-ss4n krb5kdc[21072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044977, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:36:17.633145 21316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:17.611143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:56227 (local address 127.10.25.62:42489)
0806 19:36:17.611726 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:17.611731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:17.613106 (+ 1375us) server_negotiation.cc:407] Connection header received
0806 19:36:17.616151 (+ 3045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:17.616160 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:17.616289 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:17.616439 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:17.618653 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:17.619534 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:17.621692 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:17.622121 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:17.622327 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:17.626500 (+ 4173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:17.626506 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:17.626509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:36:17.626549 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:17.629576 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:36:17.630514 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:36:17.630522 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:36:17.630531 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:36:17.630682 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:36:17.631173 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:36:17.631185 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:36:17.631188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:36:17.631476 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:17.631655 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:17.632459 (+ 804us) server_negotiation.cc:299] Negotiation successful
0806 19:36:17.632747 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":327,"threads_started":1}
I20260806 19:36:17.634413 21317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:17.611594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:42489 (local address 127.10.25.1:56227)
0806 19:36:17.612786 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:17.612872 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:17.615582 (+ 2710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:17.616782 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:17.616793 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:17.617554 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:17.618388 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:17.618412 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:17.619772 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:17.619778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:17.621414 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:17.621433 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:17.622441 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:17.622450 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:17.622726 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:17.623493 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:17.623524 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:17.626172 (+ 2648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:36:17.629788 (+ 3616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:36:17.629801 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:36:17.629822 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:36:17.630356 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:36:17.630810 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:36:17.630815 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:36:17.630819 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:36:17.631042 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:36:17.631619 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:36:17.631628 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:36:17.632040 (+ 412us) client_negotiation.cc:769] Sending connection context
0806 19:36:17.632348 (+ 308us) client_negotiation.cc:240] Negotiation successful
0806 19:36:17.632968 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":455,"threads_started":1}
I20260806 19:36:17.637123 21314 heartbeater.cc:347] Connected to a master server at 127.10.25.62:42489
I20260806 19:36:17.637733 21314 heartbeater.cc:464] Registering TS with master...
I20260806 19:36:17.639284 21314 heartbeater.cc:511] Master 127.10.25.62:42489 requested a full tablet report, sending...
I20260806 19:36:17.642809 21117 ts_manager.cc:194] Registered new tserver with Master: b9999987864444b8a1acee2b645f70ca (127.10.25.1:40663)
I20260806 19:36:17.644881 21117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:56227
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:36:17.705128 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:42489
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:36217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:36:18.142766 21320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:36:18.143100 21320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:36:18.143213 21320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:36:18.151381 21320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:36:18.151539 21320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:36:18.151665 21320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:36:18.165062 21320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:36217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:42489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:36:18.166678 21320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:36:18.168100 21320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:36:18.188401 21326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:18.188356 21327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:18.188462 21329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:36:18.189165 21320 server_base.cc:1034] running on GCE node
I20260806 19:36:18.190160 21320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:36:18.191814 21320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:36:18.193061 21320 hybrid_clock.cc:648] HybridClock initialized: now 1786044978192987 us; error 58 us; skew 500 ppm
I20260806 19:36:18.193588 21320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:36:18 dist-test-slave-ss4n krb5kdc[21072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044978, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:36:18.197693 21320 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:36:18.199863 21320 webserver.cc:460] Webserver started at http://127.10.25.2:37657/ using document root <none> and password file <none>
I20260806 19:36:18.200734 21320 fs_manager.cc:359] Metadata directory not provided
I20260806 19:36:18.200910 21320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:36:18.201268 21320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:36:18.204964 21320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "f1d5d2a003ff407fb23b1b4ae323e702"
format_stamp: "Formatted at 2026-08-06 19:36:18 on dist-test-slave-ss4n"
server_key: "2b3808a3aefbae925ee0874a6360e909"
server_key_iv: "35674f117c71c1956f9d7aa065c68f7b"
server_key_version: "encryptionkey@0"
I20260806 19:36:18.205971 21320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "f1d5d2a003ff407fb23b1b4ae323e702"
format_stamp: "Formatted at 2026-08-06 19:36:18 on dist-test-slave-ss4n"
server_key: "2b3808a3aefbae925ee0874a6360e909"
server_key_iv: "35674f117c71c1956f9d7aa065c68f7b"
server_key_version: "encryptionkey@0"
I20260806 19:36:18.215770 21320 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.001s
I20260806 19:36:18.221782 21336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:18.224108 21320 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 19:36:18.224395 21320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "f1d5d2a003ff407fb23b1b4ae323e702"
format_stamp: "Formatted at 2026-08-06 19:36:18 on dist-test-slave-ss4n"
server_key: "2b3808a3aefbae925ee0874a6360e909"
server_key_iv: "35674f117c71c1956f9d7aa065c68f7b"
server_key_version: "encryptionkey@0"
I20260806 19:36:18.224660 21320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:36:18.240365 21320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:36:18.248060 21320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:36:18.248589 21320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:36:18.249926 21320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:36:18.251487 21320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:36:18.251632 21320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:18.251785 21320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:36:18.251858 21320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:18.298642 21320 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:46349
I20260806 19:36:18.298766 21451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:46349 every 8 connection(s)
I20260806 19:36:18.300244 21320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:36:18.306424 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 21320
I20260806 19:36:18.306691 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:36:18.307246 10340 external_mini_cluster.cc:1428] Setting key 0112228984d184b874caad60494ac323
Aug 06 19:36:18 dist-test-slave-ss4n krb5kdc[21072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044978, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:36:18.337008 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:18.305344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:42773 (local address 127.10.25.62:42489)
0806 19:36:18.306033 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:18.306044 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:18.307216 (+ 1172us) server_negotiation.cc:407] Connection header received
0806 19:36:18.309659 (+ 2443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:18.309667 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:18.309979 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:18.310168 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:18.312285 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:18.313693 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:18.315706 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:18.316569 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:18.316799 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:18.325309 (+ 8510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:18.325321 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:18.325329 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:36:18.325392 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:18.331376 (+ 5984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:36:18.332971 (+ 1595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:36:18.332979 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:36:18.332983 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:36:18.333343 (+ 360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:36:18.334236 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:36:18.334242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:36:18.334245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:36:18.334646 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:18.334811 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:18.336366 (+ 1555us) server_negotiation.cc:299] Negotiation successful
0806 19:36:18.336628 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":402,"threads_started":1}
I20260806 19:36:18.338816 21455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:18.305918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:42489 (local address 127.10.25.2:42773)
0806 19:36:18.307026 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:18.307072 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:18.309098 (+ 2026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:18.310505 (+ 1407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:18.310517 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:18.311283 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:18.312015 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:18.312042 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:18.314033 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:18.314038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:18.315460 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:18.315498 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:18.316924 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:18.316930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:18.317191 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:18.318221 (+ 1030us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:18.318250 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:18.324256 (+ 6006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:36:18.331681 (+ 7425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:36:18.331698 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:36:18.331725 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:36:18.332585 (+ 860us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:36:18.333626 (+ 1041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:36:18.333636 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:36:18.333643 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:36:18.334007 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:36:18.334875 (+ 868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:36:18.334888 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:36:18.335761 (+ 873us) client_negotiation.cc:769] Sending connection context
0806 19:36:18.336126 (+ 365us) client_negotiation.cc:240] Negotiation successful
0806 19:36:18.336841 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":438,"threads_started":1}
I20260806 19:36:18.342435 21452 heartbeater.cc:347] Connected to a master server at 127.10.25.62:42489
I20260806 19:36:18.343103 21452 heartbeater.cc:464] Registering TS with master...
I20260806 19:36:18.344393 21452 heartbeater.cc:511] Master 127.10.25.62:42489 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
I20260806 19:36:18.347478 21117 ts_manager.cc:194] Registered new tserver with Master: f1d5d2a003ff407fb23b1b4ae323e702 (127.10.25.2:46349)
I20260806 19:36:18.348742 21117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:42773
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
W20260806 19:36:18.413785 10340 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.10.25.3: real 0.103s user 0.001s sys 0.031s
I20260806 19:36:18.414860 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:42489
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:36217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:36:18.650106 21314 heartbeater.cc:503] Master 127.10.25.62:42489 was elected leader, sending a full tablet report...
W20260806 19:36:18.852243 21458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:36:18.852618 21458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:36:18.852727 21458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:36:18.862905 21458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:36:18.863056 21458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:36:18.863265 21458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:36:18.877141 21458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:36217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:42489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:36:18.879721 21458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:36:18.881901 21458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:36:18.901183 21463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:18.901247 21466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:36:18.901217 21464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:36:18.902349 21458 server_base.cc:1034] running on GCE node
I20260806 19:36:18.903504 21458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:36:18.906337 21458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:36:18.908241 21458 hybrid_clock.cc:648] HybridClock initialized: now 1786044978908219 us; error 63 us; skew 500 ppm
I20260806 19:36:18.908876 21458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:36:18 dist-test-slave-ss4n krb5kdc[21072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044978, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:36:18.913976 21458 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:36:18.916467 21458 webserver.cc:460] Webserver started at http://127.10.25.3:39873/ using document root <none> and password file <none>
I20260806 19:36:18.917433 21458 fs_manager.cc:359] Metadata directory not provided
I20260806 19:36:18.917716 21458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:36:18.918246 21458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:36:18.922256 21458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "83766d866dcc4ea684f613a29a6d9050"
format_stamp: "Formatted at 2026-08-06 19:36:18 on dist-test-slave-ss4n"
server_key: "8f951bc44ca537fd7d4e19e7a43807f1"
server_key_iv: "586fe9541731d58caa27af13303194b9"
server_key_version: "encryptionkey@0"
I20260806 19:36:18.923231 21458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "83766d866dcc4ea684f613a29a6d9050"
format_stamp: "Formatted at 2026-08-06 19:36:18 on dist-test-slave-ss4n"
server_key: "8f951bc44ca537fd7d4e19e7a43807f1"
server_key_iv: "586fe9541731d58caa27af13303194b9"
server_key_version: "encryptionkey@0"
I20260806 19:36:18.931691 21458 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260806 19:36:18.936774 21473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:18.940105 21458 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.001s
I20260806 19:36:18.940405 21458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "83766d866dcc4ea684f613a29a6d9050"
format_stamp: "Formatted at 2026-08-06 19:36:18 on dist-test-slave-ss4n"
server_key: "8f951bc44ca537fd7d4e19e7a43807f1"
server_key_iv: "586fe9541731d58caa27af13303194b9"
server_key_version: "encryptionkey@0"
I20260806 19:36:18.940939 21458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:36:18.979748 21458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:36:18.986495 21458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:36:18.986903 21458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:36:18.988633 21458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:36:18.990578 21458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:36:18.990679 21458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:18.990834 21458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:36:18.990904 21458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:36:19.047497 21458 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:43785
I20260806 19:36:19.047978 21589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:43785 every 8 connection(s)
I20260806 19:36:19.049191 21458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:36:19.059262 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 21458
I20260806 19:36:19.059720 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:36:19.060446 10340 external_mini_cluster.cc:1428] Setting key a5bf31ee668f1dd7576433cd8e122ddb
Aug 06 19:36:19 dist-test-slave-ss4n krb5kdc[21072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044978, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:36:19.081712 21592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:19.053102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:59543 (local address 127.10.25.62:42489)
0806 19:36:19.053855 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:19.053860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:19.055323 (+ 1463us) server_negotiation.cc:407] Connection header received
0806 19:36:19.058433 (+ 3110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:19.058441 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:19.058585 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:19.058693 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:19.061789 (+ 3096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.063490 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.065971 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.066438 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.066617 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:19.073149 (+ 6532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:19.073156 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:19.073159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:36:19.073203 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:19.077813 (+ 4610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:36:19.078836 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:36:19.078842 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:36:19.078846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:36:19.079016 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:36:19.079549 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:36:19.079555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:36:19.079559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:36:19.079899 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:19.080103 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:19.080925 (+ 822us) server_negotiation.cc:299] Negotiation successful
0806 19:36:19.081206 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":421,"threads_started":1}
I20260806 19:36:19.082844 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:19.053816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:42489 (local address 127.10.25.3:59543)
0806 19:36:19.055095 (+ 1279us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:19.055152 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:19.057763 (+ 2611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:19.059085 (+ 1322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:19.059095 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:19.059827 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:36:19.061322 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:19.061359 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.063795 (+ 2436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.063801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:19.065751 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:19.065776 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.067333 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.067349 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:19.067693 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:19.068618 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:36:19.068648 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:36:19.072775 (+ 4127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:36:19.078063 (+ 5288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:36:19.078074 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:36:19.078091 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:36:19.078667 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:36:19.079171 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:36:19.079181 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:36:19.079184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:36:19.079407 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:36:19.080057 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:36:19.080067 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:36:19.080510 (+ 443us) client_negotiation.cc:769] Sending connection context
0806 19:36:19.080824 (+ 314us) client_negotiation.cc:240] Negotiation successful
0806 19:36:19.081527 (+ 703us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":969,"thread_start_us":758,"threads_started":1}
I20260806 19:36:19.085570 21590 heartbeater.cc:347] Connected to a master server at 127.10.25.62:42489
I20260806 19:36:19.086069 21590 heartbeater.cc:464] Registering TS with master...
I20260806 19:36:19.087333 21590 heartbeater.cc:511] Master 127.10.25.62:42489 requested a full tablet report, sending...
I20260806 19:36:19.090663 21117 ts_manager.cc:194] Registered new tserver with Master: 83766d866dcc4ea684f613a29a6d9050 (127.10.25.3:43785)
I20260806 19:36:19.092437 21117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:59543
I20260806 19:36:19.095342 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:36:19.124505 21592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:19.103994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40034 (local address 127.10.25.62:42489)
0806 19:36:19.104438 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:19.104444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:19.106322 (+ 1878us) server_negotiation.cc:407] Connection header received
0806 19:36:19.106442 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:19.106446 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:19.106534 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:19.106637 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:19.108530 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.109621 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.111408 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.111959 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.112112 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:19.115640 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:19.115650 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:19.115654 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:36:19.115724 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:19.120190 (+ 4466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:36:19.121395 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:36:19.121401 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:36:19.121404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:36:19.121792 (+ 388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:36:19.122683 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:36:19.122690 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:36:19.122693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:36:19.123234 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:19.123469 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:19.124000 (+ 531us) server_negotiation.cc:299] Negotiation successful
0806 19:36:19.124254 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260806 19:36:19.131621 21117 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:36:19.140761 21117 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:36:19.173211 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:19.162346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43110 (local address 127.10.25.2:46349)
0806 19:36:19.163223 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:19.163233 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:19.164079 (+ 846us) server_negotiation.cc:407] Connection header received
0806 19:36:19.164426 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:19.164435 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:19.164750 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:19.164941 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:36:19.165786 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.166806 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.169931 (+ 3125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.171623 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.171795 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:19.172198 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:19.172459 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 19:36:19.172818 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":352,"threads_started":1}
I20260806 19:36:19.173784 21608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:19.163107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:46349 (local address 127.0.0.1:43110)
0806 19:36:19.163943 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:19.163967 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:19.164214 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:19.165226 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:19.165230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:19.165260 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:36:19.165610 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:19.165619 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.167093 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.167100 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:19.169721 (+ 2621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:19.169748 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.171875 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.171883 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:19.172147 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:19.172208 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 19:36:19.172384 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 19:36:19.172634 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"mutex_wait_us":7,"thread_start_us":531,"threads_started":1}
I20260806 19:36:19.174204 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:19.158693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49792 (local address 127.10.25.1:40663)
0806 19:36:19.161074 (+ 2381us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:19.161099 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:19.161144 (+ 45us) server_negotiation.cc:407] Connection header received
0806 19:36:19.161282 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:19.161292 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:19.161782 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:19.162012 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:36:19.163945 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.166728 (+ 2783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.170413 (+ 3685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.172321 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.172509 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:19.172753 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:19.173596 (+ 843us) server_negotiation.cc:299] Negotiation successful
0806 19:36:19.173802 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2093,"thread_start_us":1873,"threads_started":1}
I20260806 19:36:19.174350 21603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:19.158565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:43785 (local address 127.0.0.1:47514)
0806 19:36:19.161892 (+ 3327us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:19.161908 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:19.162053 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:19.164521 (+ 2468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:19.164526 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:19.164551 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:36:19.165013 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:19.165031 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.168511 (+ 3480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.168516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:19.170767 (+ 2251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:19.170782 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.173338 (+ 2556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.173349 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:19.173642 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:19.173677 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 19:36:19.173798 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 19:36:19.173968 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3210,"thread_start_us":646,"threads_started":1}
I20260806 19:36:19.175457 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:19.158267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:40663 (local address 127.0.0.1:49792)
0806 19:36:19.159796 (+ 1529us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:19.159854 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:19.160140 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:19.162040 (+ 1900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:19.162048 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:19.162085 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:36:19.163534 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:19.163561 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.166944 (+ 3383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.166950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:19.170168 (+ 3218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:19.170184 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.172938 (+ 2754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.172946 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:19.173282 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:19.173339 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 19:36:19.174628 (+ 1289us) client_negotiation.cc:240] Negotiation successful
0806 19:36:19.174783 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1161,"mutex_wait_us":594,"thread_start_us":3301,"threads_started":1}
I20260806 19:36:19.175542 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:19.159311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47514 (local address 127.10.25.3:43785)
0806 19:36:19.161540 (+ 2229us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:19.161550 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:19.162203 (+ 653us) server_negotiation.cc:407] Connection header received
0806 19:36:19.162565 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:19.162577 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:19.163921 (+ 1344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:19.164190 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:36:19.165278 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.168312 (+ 3034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.171007 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.172501 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.172672 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:19.172926 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:19.174669 (+ 1743us) server_negotiation.cc:299] Negotiation successful
0806 19:36:19.175198 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2007,"thread_start_us":1851,"threads_started":1}
I20260806 19:36:19.179049 21524 tablet_service.cc:1467] Processing CreateTablet for tablet 5b3d48e9b2be4cdf8e6d4d5ba88d0803 (DEFAULT_TABLE table=test-table [id=ef888fccadf84113892b7f76d1d35254]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:36:19.179064 21385 tablet_service.cc:1467] Processing CreateTablet for tablet 5b3d48e9b2be4cdf8e6d4d5ba88d0803 (DEFAULT_TABLE table=test-table [id=ef888fccadf84113892b7f76d1d35254]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:36:19.180855 21385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b3d48e9b2be4cdf8e6d4d5ba88d0803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:36:19.180866 21524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b3d48e9b2be4cdf8e6d4d5ba88d0803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:36:19.181083 21246 tablet_service.cc:1467] Processing CreateTablet for tablet 5b3d48e9b2be4cdf8e6d4d5ba88d0803 (DEFAULT_TABLE table=test-table [id=ef888fccadf84113892b7f76d1d35254]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:36:19.183333 21246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b3d48e9b2be4cdf8e6d4d5ba88d0803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:36:19.201224 21609 tablet_bootstrap.cc:492] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050: Bootstrap starting.
I20260806 19:36:19.202760 21610 tablet_bootstrap.cc:492] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702: Bootstrap starting.
I20260806 19:36:19.204051 21611 tablet_bootstrap.cc:492] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca: Bootstrap starting.
I20260806 19:36:19.208050 21611 tablet_bootstrap.cc:654] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca: Neither blocks nor log segments found. Creating new log.
I20260806 19:36:19.208102 21609 tablet_bootstrap.cc:654] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050: Neither blocks nor log segments found. Creating new log.
I20260806 19:36:19.209658 21610 tablet_bootstrap.cc:654] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702: Neither blocks nor log segments found. Creating new log.
I20260806 19:36:19.210554 21611 log.cc:824] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca: Log is configured to *not* fsync() on all Append() calls
I20260806 19:36:19.210304 21609 log.cc:824] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050: Log is configured to *not* fsync() on all Append() calls
I20260806 19:36:19.212102 21610 log.cc:824] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702: Log is configured to *not* fsync() on all Append() calls
I20260806 19:36:19.215338 21611 tablet_bootstrap.cc:492] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca: No bootstrap required, opened a new log
I20260806 19:36:19.215715 21611 ts_tablet_manager.cc:1397] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260806 19:36:19.215749 21610 tablet_bootstrap.cc:492] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702: No bootstrap required, opened a new log
I20260806 19:36:19.216112 21610 ts_tablet_manager.cc:1397] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260806 19:36:19.216113 21609 tablet_bootstrap.cc:492] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050: No bootstrap required, opened a new log
I20260806 19:36:19.216665 21609 ts_tablet_manager.cc:1397] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050: Time spent bootstrapping tablet: real 0.016s user 0.012s sys 0.000s
I20260806 19:36:19.228636 21609 raft_consensus.cc:348] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9999987864444b8a1acee2b645f70ca" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 40663 } } peers { permanent_uuid: "83766d866dcc4ea684f613a29a6d9050" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43785 } } peers { permanent_uuid: "f1d5d2a003ff407fb23b1b4ae323e702" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46349 } }
I20260806 19:36:19.228636 21611 raft_consensus.cc:348] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9999987864444b8a1acee2b645f70ca" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 40663 } } peers { permanent_uuid: "83766d866dcc4ea684f613a29a6d9050" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43785 } } peers { permanent_uuid: "f1d5d2a003ff407fb23b1b4ae323e702" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46349 } }
I20260806 19:36:19.229336 21611 raft_consensus.cc:374] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:36:19.229387 21609 raft_consensus.cc:374] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:36:19.229547 21611 raft_consensus.cc:729] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9999987864444b8a1acee2b645f70ca, State: Initialized, Role: FOLLOWER
I20260806 19:36:19.229630 21609 raft_consensus.cc:729] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83766d866dcc4ea684f613a29a6d9050, State: Initialized, Role: FOLLOWER
I20260806 19:36:19.230026 21610 raft_consensus.cc:348] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9999987864444b8a1acee2b645f70ca" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 40663 } } peers { permanent_uuid: "83766d866dcc4ea684f613a29a6d9050" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43785 } } peers { permanent_uuid: "f1d5d2a003ff407fb23b1b4ae323e702" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46349 } }
I20260806 19:36:19.230625 21610 raft_consensus.cc:374] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:36:19.230731 21610 raft_consensus.cc:729] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d5d2a003ff407fb23b1b4ae323e702, State: Initialized, Role: FOLLOWER
I20260806 19:36:19.230418 21611 consensus_queue.cc:260] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9999987864444b8a1acee2b645f70ca" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 40663 } } peers { permanent_uuid: "83766d866dcc4ea684f613a29a6d9050" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43785 } } peers { permanent_uuid: "f1d5d2a003ff407fb23b1b4ae323e702" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46349 } }
I20260806 19:36:19.230418 21609 consensus_queue.cc:260] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9999987864444b8a1acee2b645f70ca" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 40663 } } peers { permanent_uuid: "83766d866dcc4ea684f613a29a6d9050" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43785 } } peers { permanent_uuid: "f1d5d2a003ff407fb23b1b4ae323e702" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46349 } }
I20260806 19:36:19.231488 21610 consensus_queue.cc:260] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9999987864444b8a1acee2b645f70ca" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 40663 } } peers { permanent_uuid: "83766d866dcc4ea684f613a29a6d9050" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43785 } } peers { permanent_uuid: "f1d5d2a003ff407fb23b1b4ae323e702" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46349 } }
I20260806 19:36:19.233640 21452 heartbeater.cc:503] Master 127.10.25.62:42489 was elected leader, sending a full tablet report...
I20260806 19:36:19.234421 21610 ts_tablet_manager.cc:1428] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702: Time spent starting tablet: real 0.018s user 0.015s sys 0.004s
I20260806 19:36:19.236910 21611 ts_tablet_manager.cc:1428] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca: Time spent starting tablet: real 0.021s user 0.014s sys 0.008s
I20260806 19:36:19.237061 21609 ts_tablet_manager.cc:1428] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050: Time spent starting tablet: real 0.020s user 0.018s sys 0.004s
I20260806 19:36:19.237646 21590 heartbeater.cc:503] Master 127.10.25.62:42489 was elected leader, sending a full tablet report...
W20260806 19:36:19.301620 21591 tablet.cc:2369] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:36:19.303963 21453 tablet.cc:2369] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:36:19.362074 21315 tablet.cc:2369] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:36:19.574182 21617 raft_consensus.cc:482] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:36:19.574682 21617 raft_consensus.cc:504] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9999987864444b8a1acee2b645f70ca" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 40663 } } peers { permanent_uuid: "83766d866dcc4ea684f613a29a6d9050" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43785 } } peers { permanent_uuid: "f1d5d2a003ff407fb23b1b4ae323e702" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46349 } }
I20260806 19:36:19.576725 21617 leader_election.cc:288] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9999987864444b8a1acee2b645f70ca (127.10.25.1:40663), 83766d866dcc4ea684f613a29a6d9050 (127.10.25.3:43785)
I20260806 19:36:19.585131 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:19.577357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:42613 (local address 127.10.25.3:43785)
0806 19:36:19.577595 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:19.577603 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:19.578768 (+ 1165us) server_negotiation.cc:407] Connection header received
0806 19:36:19.579007 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:19.579013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:19.579099 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:19.579214 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:36:19.579856 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.580605 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.582904 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.584065 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.584244 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:19.584362 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:19.584709 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 19:36:19.584849 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 19:36:19.585222 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:19.577304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:43785 (local address 127.10.25.2:42613)
0806 19:36:19.578579 (+ 1275us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:19.578598 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:19.578835 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:19.579304 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:19.579311 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:19.579345 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:36:19.579686 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:19.579697 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.580786 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.580791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:19.582707 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:19.582726 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.584338 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.584347 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:19.584515 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:19.584542 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 19:36:19.584678 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 19:36:19.584840 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1134,"thread_start_us":657,"threads_started":1}
I20260806 19:36:19.586529 21545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3d48e9b2be4cdf8e6d4d5ba88d0803" candidate_uuid: "f1d5d2a003ff407fb23b1b4ae323e702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83766d866dcc4ea684f613a29a6d9050" is_pre_election: true
I20260806 19:36:19.587229 21545 raft_consensus.cc:2454] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1d5d2a003ff407fb23b1b4ae323e702 in term 0.
I20260806 19:36:19.587530 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:19.577336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:40663 (local address 127.10.25.2:36425)
0806 19:36:19.578882 (+ 1546us) negotiation.cc:107] Waiting for socket to connect
0806 19:36:19.578920 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 19:36:19.579195 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:36:19.579604 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:36:19.579609 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:36:19.579646 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:36:19.580391 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:19.580409 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.582044 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.582052 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:19.584779 (+ 2727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:36:19.584798 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.586725 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.586730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:36:19.586941 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:19.586970 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 19:36:19.587053 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 19:36:19.587207 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1381,"thread_start_us":593,"threads_started":1}
I20260806 19:36:19.587721 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:19.577308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:36425 (local address 127.10.25.1:40663)
0806 19:36:19.577703 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:19.577713 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:19.579112 (+ 1399us) server_negotiation.cc:407] Connection header received
0806 19:36:19.579382 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:19.579387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:19.579467 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:19.579582 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:36:19.580714 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.581827 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.585021 (+ 3194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.586452 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.586633 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:19.586770 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:19.587113 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 19:36:19.587319 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260806 19:36:19.588110 21339 leader_election.cc:302] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83766d866dcc4ea684f613a29a6d9050, f1d5d2a003ff407fb23b1b4ae323e702; no voters:
I20260806 19:36:19.588577 21617 raft_consensus.cc:2781] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:36:19.588704 21617 raft_consensus.cc:482] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:36:19.588835 21617 raft_consensus.cc:3037] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:36:19.588734 21266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3d48e9b2be4cdf8e6d4d5ba88d0803" candidate_uuid: "f1d5d2a003ff407fb23b1b4ae323e702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9999987864444b8a1acee2b645f70ca" is_pre_election: true
I20260806 19:36:19.589381 21266 raft_consensus.cc:2454] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1d5d2a003ff407fb23b1b4ae323e702 in term 0.
I20260806 19:36:19.591475 21617 raft_consensus.cc:504] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9999987864444b8a1acee2b645f70ca" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 40663 } } peers { permanent_uuid: "83766d866dcc4ea684f613a29a6d9050" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43785 } } peers { permanent_uuid: "f1d5d2a003ff407fb23b1b4ae323e702" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46349 } }
I20260806 19:36:19.592440 21617 leader_election.cc:288] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [CANDIDATE]: Term 1 election: Requested vote from peers b9999987864444b8a1acee2b645f70ca (127.10.25.1:40663), 83766d866dcc4ea684f613a29a6d9050 (127.10.25.3:43785)
I20260806 19:36:19.593236 21266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3d48e9b2be4cdf8e6d4d5ba88d0803" candidate_uuid: "f1d5d2a003ff407fb23b1b4ae323e702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9999987864444b8a1acee2b645f70ca"
I20260806 19:36:19.593223 21545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3d48e9b2be4cdf8e6d4d5ba88d0803" candidate_uuid: "f1d5d2a003ff407fb23b1b4ae323e702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83766d866dcc4ea684f613a29a6d9050"
I20260806 19:36:19.593533 21545 raft_consensus.cc:3037] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:36:19.593572 21266 raft_consensus.cc:3037] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:36:19.596136 21266 raft_consensus.cc:2454] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1d5d2a003ff407fb23b1b4ae323e702 in term 1.
I20260806 19:36:19.596184 21545 raft_consensus.cc:2454] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1d5d2a003ff407fb23b1b4ae323e702 in term 1.
I20260806 19:36:19.596764 21338 leader_election.cc:302] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9999987864444b8a1acee2b645f70ca, f1d5d2a003ff407fb23b1b4ae323e702; no voters:
I20260806 19:36:19.597162 21617 raft_consensus.cc:2781] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:36:19.597792 21617 raft_consensus.cc:686] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [term 1 LEADER]: Becoming Leader. State: Replica: f1d5d2a003ff407fb23b1b4ae323e702, State: Running, Role: LEADER
I20260806 19:36:19.598361 21617 consensus_queue.cc:237] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9999987864444b8a1acee2b645f70ca" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 40663 } } peers { permanent_uuid: "83766d866dcc4ea684f613a29a6d9050" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43785 } } peers { permanent_uuid: "f1d5d2a003ff407fb23b1b4ae323e702" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46349 } }
I20260806 19:36:19.604485 21115 catalog_manager.cc:5495] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1d5d2a003ff407fb23b1b4ae323e702 (127.10.25.2). New cstate: current_term: 1 leader_uuid: "f1d5d2a003ff407fb23b1b4ae323e702" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9999987864444b8a1acee2b645f70ca" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 40663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83766d866dcc4ea684f613a29a6d9050" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1d5d2a003ff407fb23b1b4ae323e702" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46349 } health_report { overall_health: HEALTHY } } }
I20260806 19:36:19.650511 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:19.639789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43116 (local address 127.10.25.2:46349)
0806 19:36:19.640018 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:19.640024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:19.640543 (+ 519us) server_negotiation.cc:407] Connection header received
0806 19:36:19.640703 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:19.640708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:19.640809 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:19.640997 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:36:19.641683 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.642583 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.644926 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.646026 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.646176 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:19.647266 (+ 1090us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:36:19.648710 (+ 1444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:36:19.649006 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:19.649938 (+ 932us) server_negotiation.cc:299] Negotiation successful
0806 19:36:19.650053 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 19:36:19.669438 21545 raft_consensus.cc:1261] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050 [term 1 FOLLOWER]: Refusing update from remote peer f1d5d2a003ff407fb23b1b4ae323e702: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:36:19.669975 21266 raft_consensus.cc:1261] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca [term 1 FOLLOWER]: Refusing update from remote peer f1d5d2a003ff407fb23b1b4ae323e702: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:36:19.671324 21620 consensus_queue.cc:1035] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83766d866dcc4ea684f613a29a6d9050" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:36:19.671846 21617 consensus_queue.cc:1035] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9999987864444b8a1acee2b645f70ca" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 40663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:36:19.732553 21117 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:36:19.732956 21117 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:36:19.743307 21117 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 274a377a5dca4cd8a6a33a103ed44068: Sending DeleteTablet for 3 replicas of tablet 5b3d48e9b2be4cdf8e6d4d5ba88d0803
I20260806 19:36:19.745321 21524 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b3d48e9b2be4cdf8e6d4d5ba88d0803 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:36:19 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:47514
I20260806 19:36:19.745333 21246 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b3d48e9b2be4cdf8e6d4d5ba88d0803 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:36:19 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:49792
I20260806 19:36:19.749505 21385 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b3d48e9b2be4cdf8e6d4d5ba88d0803 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:36:19 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:43110
I20260806 19:36:19.752077 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 21181
I20260806 19:36:19.761830 21634 tablet_replica.cc:330] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050: stopping tablet replica
I20260806 19:36:19.762399 21634 raft_consensus.cc:2229] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:36:19.763258 21634 raft_consensus.cc:2258] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:36:19.765892 21634 ts_tablet_manager.cc:1905] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:36:19.776139 21634 ts_tablet_manager.cc:1918] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:36:19.776381 21634 log.cc:1196] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/5b3d48e9b2be4cdf8e6d4d5ba88d0803
I20260806 19:36:19.776988 21634 ts_tablet_manager.cc:1939] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P 83766d866dcc4ea684f613a29a6d9050: Deleting consensus metadata
I20260806 19:36:19.780735 21103 catalog_manager.cc:4841] TS 83766d866dcc4ea684f613a29a6d9050 (127.10.25.3:43785): tablet 5b3d48e9b2be4cdf8e6d4d5ba88d0803 (table test-table [id=ef888fccadf84113892b7f76d1d35254]) successfully deleted
I20260806 19:36:19.782810 21636 tablet_replica.cc:330] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702: stopping tablet replica
I20260806 19:36:19.784817 21635 tablet_replica.cc:330] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca: stopping tablet replica
I20260806 19:36:19.785362 21635 raft_consensus.cc:2229] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:36:19.786005 21635 raft_consensus.cc:2258] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:36:19.791160 21636 raft_consensus.cc:2229] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:36:19.793120 21636 raft_consensus.cc:2258] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:36:19.796694 21635 ts_tablet_manager.cc:1905] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 19:36:19 dist-test-slave-ss4n krb5kdc[21072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:36:19.804484 21636 ts_tablet_manager.cc:1905] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:36:19.814707 21635 ts_tablet_manager.cc:1918] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:36:19.814942 21635 log.cc:1196] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/5b3d48e9b2be4cdf8e6d4d5ba88d0803
I20260806 19:36:19.815559 21635 ts_tablet_manager.cc:1939] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P b9999987864444b8a1acee2b645f70ca: Deleting consensus metadata
I20260806 19:36:19.817931 21102 catalog_manager.cc:4841] TS b9999987864444b8a1acee2b645f70ca (127.10.25.1:40663): tablet 5b3d48e9b2be4cdf8e6d4d5ba88d0803 (table test-table [id=ef888fccadf84113892b7f76d1d35254]) successfully deleted
I20260806 19:36:19.820390 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:19.753725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49800 (local address 127.10.25.1:40663)
0806 19:36:19.754012 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:19.754022 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:19.775072 (+ 21050us) server_negotiation.cc:407] Connection header received
0806 19:36:19.781771 (+ 6699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:19.781778 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:19.781914 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:19.782008 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:19.788868 (+ 6860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.790258 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.794496 (+ 4238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:19.795490 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:19.795632 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:19.806487 (+ 10855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:19.806514 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:19.806537 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:36:19.806602 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:19.814140 (+ 7538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:36:19.815801 (+ 1661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:36:19.815817 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:36:19.815824 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:36:19.816083 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:36:19.816761 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:36:19.816767 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:36:19.816771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:36:19.817351 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:19.817767 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:19.819351 (+ 1584us) server_negotiation.cc:299] Negotiation successful
0806 19:36:19.819834 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 19:36:19.820924 21636 ts_tablet_manager.cc:1918] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:36:19.821151 21636 log.cc:1196] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/5b3d48e9b2be4cdf8e6d4d5ba88d0803
I20260806 19:36:19.821537 21309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:36:19.821849 21636 ts_tablet_manager.cc:1939] T 5b3d48e9b2be4cdf8e6d4d5ba88d0803 P f1d5d2a003ff407fb23b1b4ae323e702: Deleting consensus metadata
I20260806 19:36:19.824818 21102 catalog_manager.cc:4841] TS f1d5d2a003ff407fb23b1b4ae323e702 (127.10.25.2:46349): tablet 5b3d48e9b2be4cdf8e6d4d5ba88d0803 (table test-table [id=ef888fccadf84113892b7f76d1d35254]) successfully deleted
I20260806 19:36:19.982843 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 21181
I20260806 19:36:20.010317 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 21320
Aug 06 19:36:20 dist-test-slave-ss4n krb5kdc[21072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:36:20.050952 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:20.011807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43126 (local address 127.10.25.2:46349)
0806 19:36:20.011998 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:20.012002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:20.012022 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:36:20.012426 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:20.012433 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:20.012537 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:20.012679 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:20.016651 (+ 3972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:20.019105 (+ 2454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:20.023752 (+ 4647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:20.024758 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:20.024956 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:20.035349 (+ 10393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:20.035374 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:20.035394 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:36:20.035471 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:20.045264 (+ 9793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:36:20.047248 (+ 1984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:36:20.047263 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:36:20.047268 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:36:20.047711 (+ 443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:36:20.048590 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:36:20.048599 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:36:20.048603 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:36:20.049251 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:20.049733 (+ 482us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:20.050131 (+ 398us) server_negotiation.cc:299] Negotiation successful
0806 19:36:20.050445 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 19:36:20.052264 21447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:36:20.205528 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 21320
I20260806 19:36:20.232270 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 21458
Aug 06 19:36:20 dist-test-slave-ss4n krb5kdc[21072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:36:20.256057 21641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:36:20.233202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47518 (local address 127.10.25.3:43785)
0806 19:36:20.234093 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0806 19:36:20.234103 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:36:20.234137 (+ 34us) server_negotiation.cc:407] Connection header received
0806 19:36:20.234232 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:36:20.234239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:36:20.234337 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:36:20.234505 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:36:20.236421 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:20.237618 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:20.239324 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:36:20.240183 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:36:20.240469 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:36:20.246651 (+ 6182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:36:20.246676 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:36:20.246700 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:36:20.246764 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:36:20.250992 (+ 4228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:36:20.252421 (+ 1429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:36:20.252436 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:36:20.252440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:36:20.252762 (+ 322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:36:20.253610 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:36:20.253616 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:36:20.253619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:36:20.254288 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:36:20.254665 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0806 19:36:20.255153 (+ 488us) server_negotiation.cc:299] Negotiation successful
0806 19:36:20.255561 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":730,"thread_start_us":465,"threads_started":1}
I20260806 19:36:20.256976 21585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:36:20.371096 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 21458
I20260806 19:36:20.396660 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 21085
I20260806 19:36:20.398964 21147 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:36:20.530457 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 21085
2026-08-06T19:36:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (6142 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:36:20 dist-test-slave-ss4n krb5kdc[21647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:36:20 dist-test-slave-ss4n krb5kdc[21647](info): set up 2 sockets
Aug 06 19:36:20 dist-test-slave-ss4n krb5kdc[21647](info): commencing operation
krb5kdc: starting...
W20260806 19:36:21.945351 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.359s user 0.004s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:36:22 dist-test-slave-ss4n krb5kdc[21647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786044982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:36:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:36:22Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.25.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.25.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.25.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/rangeradmin_127.10.25.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.25.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/rangeradmin_127.10.25.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.25.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.25.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.25.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/rangerlookup_127.10.25.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.25.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/rangerlookup_127.10.25.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.25.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.25.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.25.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/HTTP_127.10.25.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.25.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/HTTP_127.10.25.20@KRBTEST.COM.keytab.
I20260806 19:36:22.534802 10340 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskduu4iA/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/postgres -l logfile start
2026-08-06 19:36:26.810 UTC [21678] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 19:36:26.810 UTC [21678] LOG: listening on IPv4 address "127.10.25.20", port 58241
2026-08-06 19:36:26.816 UTC [21678] LOG: listening on Unix socket "/tmp/.s.PGSQL.58241"
2026-08-06 19:36:26.850 UTC [21681] LOG: database system was shut down at 2026-08-06 19:36:23 UTC
2026-08-06 19:36:26.855 UTC [21678] LOG: database system is ready to accept connections
I20260806 19:36:27.980481 10340 mini_postgres.cc:96] Postgres bound to 58241
2026-08-06 19:36:28.008 UTC [21689] FATAL: database "slave" does not exist
127.10.25.20:58241 - accepting connections
I20260806 19:36:28.009413 10340 mini_ranger.cc:161] Starting Ranger...
I20260806 19:36:28.045446 10340 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 19:36:28.937069 10340 mini_ranger.cc:87] Created ranger Postgres database
I20260806 19:36:28.937274 10340 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-admin
2026-08-06 19:36:29,320 [I] DB FLAVOR :POSTGRES
2026-08-06 19:36:29,322 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 19:36:29,322 [I] Checking connection..
2026-08-06 19:36:29,322 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 19:36:29,627 [I] Checking connection passed.
2026-08-06 19:36:29,628 [I] --------- Verifying version history table ---------
2026-08-06 19:36:29,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 19:36:29,963 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 19:36:29,964 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 19:36:30,293 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 19:36:30,294 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 19:36:30,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 19:36:30.621 UTC [21753] WARNING: there is no transaction in progress
2026-08-06 19:36:30,630 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 19:36:30,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 19:36:30,955 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 19:36:30,955 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 19:36:30,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 19:36:31,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 19:36:31,591 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ss4n.c.gcp-upstream.internal','N') ;"
2026-08-06 19:36:31,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 19:36:32,284 [I] Table x_portal_user does not exist in database ranger
2026-08-06 19:36:32,285 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 19:36:32,653 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 19:36:32,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 19:36:33,009 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 19:36:33,389 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 19:36:33,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 19:36:35.137 UTC [21865] WARNING: there is no transaction in progress
2026-08-06 19:36:35.175 UTC [21865] WARNING: there is no transaction in progress
2026-08-06 19:36:35.208 UTC [21865] WARNING: there is no transaction in progress
2026-08-06 19:36:35.238 UTC [21865] WARNING: there is no transaction in progress
2026-08-06 19:36:35.270 UTC [21865] WARNING: there is no transaction in progress
2026-08-06 19:36:35.310 UTC [21865] WARNING: there is no transaction in progress
2026-08-06 19:36:35.431 UTC [21865] WARNING: there is no transaction in progress
2026-08-06 19:36:35.445 UTC [21865] WARNING: there is no transaction in progress
2026-08-06 19:36:35.475 UTC [21865] WARNING: there is no transaction in progress
2026-08-06 19:36:35.492 UTC [21865] WARNING: there is no transaction in progress
2026-08-06 19:36:35.510 UTC [21865] WARNING: there is no transaction in progress
2026-08-06 19:36:35.531 UTC [21865] WARNING: there is no transaction in progress
2026-08-06 19:36:35.546 UTC [21865] WARNING: there is no transaction in progress
2026-08-06 19:36:35.559 UTC [21865] WARNING: there is no transaction in progress
2026-08-06 19:36:35.573 UTC [21865] WARNING: there is no transaction in progress
2026-08-06 19:36:35.591 UTC [21865] WARNING: there is no transaction in progress
2026-08-06 19:36:37,053 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 19:36:37,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 19:36:37,421 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 19:36:37,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 19:36:37,799 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 19:36:37,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 19:36:38,135 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 19:36:38,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 19:36:38,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 19:36:38,830 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ss4n.c.gcp-upstream.internal';"
2026-08-06 19:36:39,201 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 19:36:39,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 19:36:39,602 [I] DB_PATCHES have already been applied
I20260806 19:36:39.612344 10340 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-admin:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskduu4iA/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 19:36:39.612553 10340 mini_ranger.cc:193] Using host: 127.10.25.20
I20260806 19:36:39.640684 10340 mini_ranger.cc:227] Ranger admin URL: http://127.10.25.20:38119
Aug 06, 2026 7:36:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 7:36:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 7:36:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 7:36:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 7:36:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 7:36:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 7:36:40 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 19:36:41.238725 10340 mini_ranger.cc:160] Time spent starting Ranger: real 13.229s user 0.003s sys 0.076s
Aug 06, 2026 7:37:11 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 7:37:12 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 7:37:12 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 7:37:12 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 7:37:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 7:37:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 7:37:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 7:37:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 7:37:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 7:37:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 7:37:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 7:37:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 7:37:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 7:37:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 7:37:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 7:37:12 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 7:37:12 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 7:37:13 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 19:37:14.015272 10340 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.25.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.25.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.25.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/rangerkms_127.10.25.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.25.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/rangerkms_127.10.25.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.25.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.25.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.25.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/HTTP_127.10.25.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.25.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/HTTP_127.10.25.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 19:37:14 dist-test-slave-ss4n krb5kdc[21647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045034, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 19:37:14.308439 10340 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 19:37:14.351579 10340 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 19:37:15.613124 10340 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 19:37:15.613301 10340 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-kms
I20260806 19:37:15.613601 10340 mini_ranger_kms.cc:227] Using postgres at 127.10.25.20:58241
2026-08-06 19:37:16,484 [I] DB FLAVOR :POSTGRES
2026-08-06 19:37:16,484 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 19:37:16,484 [I] Checking connection
2026-08-06 19:37:16,484 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 19:37:16,961 [I] connection success
2026-08-06 19:37:16,962 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 19:37:16,962 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 19:37:16,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 19:37:17,371 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 19:37:17,371 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 19:37:17,372 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 19:37:17,372 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:58241/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 19:37:17,773 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 19:37:18.455348 10340 mini_ranger_kms.cc:310] Created kms service
I20260806 19:37:18.683368 10340 mini_ranger_kms.cc:326] Created kudu user
I20260806 19:37:18.747189 10340 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 19:37:18 dist-test-slave-ss4n krb5kdc[21647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045034, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
I20260806 19:37:19.365185 10340 mini_ranger_kms.cc:384] Added ranger policy
I20260806 19:37:19.370790 10340 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-kms:/tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 19:37:19.370890 10340 mini_ranger_kms.cc:240] Using host: 127.10.25.20
I20260806 19:37:19.404897 10340 mini_ranger_kms.cc:276] Ranger KMS PID: 22052
I20260806 19:37:19.405170 10340 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.25.20:39867
Aug 06, 2026 7:37:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 7:37:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 7:37:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 7:37:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 7:37:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 7:37:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 19:37:22.092202 10340 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 7.784s user 0.282s sys 0.620s
I20260806 19:37:22.092618 10340 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 19:37:22.092871 10340 mini_ranger_kms.cc:401] 127.10.25.20:39867/kms/v1/keys
Aug 06 19:37:22 dist-test-slave-ss4n krb5kdc[21647](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786045042, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.25.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 19:37:22 dist-test-slave-ss4n krb5kdc[21647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045042, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.25.20@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
Aug 06 19:37:23 dist-test-slave-ss4n krb5kdc[21647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045042, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.25.20@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
Aug 06 19:37:25 dist-test-slave-ss4n krb5kdc[21647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:37:26.092329 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:36075
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:36075
--ranger_config_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.25.20:39867/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:37:26.531220 22104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:37:26.531581 22104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:37:26.531740 22104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:37:26.540294 22104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:37:26.540486 22104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:37:26.540623 22104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:37:26.540740 22104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:37:26.540827 22104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:37:26.553196 22104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.25.20:39867/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:36075
--ranger_config_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:36075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:37:26.554790 22104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:37:26.556320 22104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:37:26.572958 22112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:37:26.573004 22109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:37:26.572858 22110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:37:26.572875 22104 server_base.cc:1034] running on GCE node
I20260806 19:37:26.574404 22104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:37:26.576128 22104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:37:26.577421 22104 hybrid_clock.cc:648] HybridClock initialized: now 1786045046577384 us; error 70 us; skew 500 ppm
I20260806 19:37:26.577914 22104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:37:26 dist-test-slave-ss4n krb5kdc[21647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045046, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:37:26.581805 22104 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:37:26.583949 22104 webserver.cc:460] Webserver started at http://127.10.25.62:43865/ using document root <none> and password file <none>
I20260806 19:37:26.584766 22104 fs_manager.cc:359] Metadata directory not provided
I20260806 19:37:26.584937 22104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:37:26.585330 22104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 19:37:26 dist-test-slave-ss4n krb5kdc[21647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045046, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
I20260806 19:37:26.650177 22104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "bab72fb52b46492ba1bbeed5009739b1"
format_stamp: "Formatted at 2026-08-06 19:37:26 on dist-test-slave-ss4n"
server_key: "8207c55e03656b525b8d29382ef4ddd9"
server_key_iv: "8384f92d24e482626b75dd29a6f5089a"
server_key_version: "kuduclusterkey@0"
I20260806 19:37:26.651181 22104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "bab72fb52b46492ba1bbeed5009739b1"
format_stamp: "Formatted at 2026-08-06 19:37:26 on dist-test-slave-ss4n"
server_key: "8207c55e03656b525b8d29382ef4ddd9"
server_key_iv: "8384f92d24e482626b75dd29a6f5089a"
server_key_version: "kuduclusterkey@0"
I20260806 19:37:26.658375 22104 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 19:37:26.678917 22119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:37:26.680748 22104 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 19:37:26.680982 22104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "bab72fb52b46492ba1bbeed5009739b1"
format_stamp: "Formatted at 2026-08-06 19:37:26 on dist-test-slave-ss4n"
server_key: "8207c55e03656b525b8d29382ef4ddd9"
server_key_iv: "8384f92d24e482626b75dd29a6f5089a"
server_key_version: "kuduclusterkey@0"
I20260806 19:37:26.681262 22104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:37:26.705895 22104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:37:26.710739 22104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:37:26.711117 22104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:37:26.737357 22104 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:36075
I20260806 19:37:26.737773 22172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:36075 every 8 connection(s)
I20260806 19:37:26.739375 22104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:37:26.744252 22173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:37:26.748479 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 22104
I20260806 19:37:26.748773 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:37:26.749584 10340 external_mini_cluster.cc:1428] Setting key a82def74294f417871a7031204def7f3
I20260806 19:37:26.760562 22173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1: Bootstrap starting.
I20260806 19:37:26.766084 22173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1: Neither blocks nor log segments found. Creating new log.
Aug 06 19:37:26 dist-test-slave-ss4n krb5kdc[21647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:37:26.768790 22173 log.cc:824] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1: Log is configured to *not* fsync() on all Append() calls
I20260806 19:37:26.774076 22173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1: No bootstrap required, opened a new log
I20260806 19:37:26.780622 22176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:26.752381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41608 (local address 127.10.25.62:36075)
0806 19:37:26.753264 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0806 19:37:26.753278 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:37:26.753323 (+ 45us) server_negotiation.cc:407] Connection header received
0806 19:37:26.755266 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:37:26.755294 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:37:26.755756 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:37:26.756148 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:37:26.758211 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:26.759893 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:26.762064 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:26.762946 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:26.763119 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:26.770257 (+ 7138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:37:26.770274 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:37:26.770305 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:37:26.770373 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:37:26.774784 (+ 4411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:37:26.776041 (+ 1257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:37:26.776052 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:37:26.776061 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:37:26.776294 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:37:26.776897 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:37:26.776905 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:37:26.776907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:37:26.777816 (+ 909us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:37:26.778112 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0806 19:37:26.778873 (+ 761us) server_negotiation.cc:299] Negotiation successful
0806 19:37:26.779375 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":420,"threads_started":1}
I20260806 19:37:26.786048 22173 raft_consensus.cc:348] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bab72fb52b46492ba1bbeed5009739b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36075 } }
I20260806 19:37:26.786540 22173 raft_consensus.cc:374] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:37:26.786710 22173 raft_consensus.cc:729] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab72fb52b46492ba1bbeed5009739b1, State: Initialized, Role: FOLLOWER
I20260806 19:37:26.787530 22173 consensus_queue.cc:260] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bab72fb52b46492ba1bbeed5009739b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36075 } }
I20260806 19:37:26.787926 22173 raft_consensus.cc:388] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:37:26.788121 22173 raft_consensus.cc:482] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:37:26.788338 22173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:37:26.791280 22173 raft_consensus.cc:504] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bab72fb52b46492ba1bbeed5009739b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36075 } }
I20260806 19:37:26.791946 22173 leader_election.cc:302] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bab72fb52b46492ba1bbeed5009739b1; no voters:
I20260806 19:37:26.792721 22173 leader_election.cc:288] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:37:26.792896 22178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:37:26.795017 22178 raft_consensus.cc:686] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [term 1 LEADER]: Becoming Leader. State: Replica: bab72fb52b46492ba1bbeed5009739b1, State: Running, Role: LEADER
I20260806 19:37:26.795444 22173 sys_catalog.cc:564] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:37:26.795806 22178 consensus_queue.cc:237] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bab72fb52b46492ba1bbeed5009739b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36075 } }
I20260806 19:37:26.800868 22179 sys_catalog.cc:455] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bab72fb52b46492ba1bbeed5009739b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bab72fb52b46492ba1bbeed5009739b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36075 } } }
I20260806 19:37:26.801417 22179 sys_catalog.cc:458] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [sys.catalog]: This master's current role is: LEADER
I20260806 19:37:26.802030 22180 sys_catalog.cc:455] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bab72fb52b46492ba1bbeed5009739b1. Latest consensus state: current_term: 1 leader_uuid: "bab72fb52b46492ba1bbeed5009739b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bab72fb52b46492ba1bbeed5009739b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36075 } } }
I20260806 19:37:26.802590 22180 sys_catalog.cc:458] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1 [sys.catalog]: This master's current role is: LEADER
I20260806 19:37:26.814512 22186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:37:26.823253 22186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:37:26.824470 22173 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 19:37:26.843839 22186 catalog_manager.cc:1347] Generated new cluster ID: 6dcb694ecb0f48e4a335aac0820d8482
I20260806 19:37:26.844035 22186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:37:26.856492 22186 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:37:26.860674 22186 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:37:26.877177 22186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1: Generated new TSK 0
I20260806 19:37:26.878644 22186 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:37:27.868296 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.25.20:39867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36075
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:37:28 dist-test-slave-ss4n krb5kdc[21647](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786045048, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 19:37:28.438903 22220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:37:28.439394 22220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:37:28.439610 22220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:37:28.451460 22220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:37:28.451658 22220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:37:28.451905 22220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:37:28.464619 22220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.25.20:39867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:37:28.466773 22220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:37:28.468694 22220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:37:28.497054 22231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:37:28.497262 22230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:37:28.512266 22233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:37:28.513465 22220 server_base.cc:1034] running on GCE node
I20260806 19:37:28.514344 22220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:37:28.515540 22220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:37:28.516916 22220 hybrid_clock.cc:648] HybridClock initialized: now 1786045048516836 us; error 77 us; skew 500 ppm
I20260806 19:37:28.517330 22220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:37:28 dist-test-slave-ss4n krb5kdc[21647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045048, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:37:28.526746 22220 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:37:28.530256 22220 webserver.cc:460] Webserver started at http://127.10.25.1:40313/ using document root <none> and password file <none>
I20260806 19:37:28.531316 22220 fs_manager.cc:359] Metadata directory not provided
I20260806 19:37:28.531505 22220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:37:28.531913 22220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 19:37:28 dist-test-slave-ss4n krb5kdc[21647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045048, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
I20260806 19:37:28.552764 22220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "d1a9aa6111454428bd610813e0da907c"
format_stamp: "Formatted at 2026-08-06 19:37:28 on dist-test-slave-ss4n"
server_key: "229c10aec7b6f9bd146b95d0956aed95"
server_key_iv: "562a50384cc905448995add8f83184e2"
server_key_version: "kuduclusterkey@0"
I20260806 19:37:28.554206 22220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "d1a9aa6111454428bd610813e0da907c"
format_stamp: "Formatted at 2026-08-06 19:37:28 on dist-test-slave-ss4n"
server_key: "229c10aec7b6f9bd146b95d0956aed95"
server_key_iv: "562a50384cc905448995add8f83184e2"
server_key_version: "kuduclusterkey@0"
I20260806 19:37:28.563969 22220 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.008s sys 0.000s
I20260806 19:37:28.608563 22240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:37:28.614329 22220 fs_manager.cc:725] Time spent opening block manager: real 0.023s user 0.009s sys 0.010s
I20260806 19:37:28.614734 22220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "d1a9aa6111454428bd610813e0da907c"
format_stamp: "Formatted at 2026-08-06 19:37:28 on dist-test-slave-ss4n"
server_key: "229c10aec7b6f9bd146b95d0956aed95"
server_key_iv: "562a50384cc905448995add8f83184e2"
server_key_version: "kuduclusterkey@0"
I20260806 19:37:28.615262 22220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:37:28.679946 22220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:37:28.688591 22220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:37:28.689105 22220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:37:28.691846 22220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:37:28.694190 22220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:37:28.694317 22220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:37:28.694522 22220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:37:28.694624 22220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:37:28 dist-test-slave-ss4n krb5kdc[21647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045048, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
I20260806 19:37:28.906843 22220 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:33621
I20260806 19:37:28.906901 22354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:33621 every 8 connection(s)
I20260806 19:37:28.909046 22220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:37:28.910567 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 22220
I20260806 19:37:28.910880 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:37:28.912175 10340 external_mini_cluster.cc:1428] Setting key 08b63a84ed9cd3973e41bffabf40c7bf
Aug 06 19:37:28 dist-test-slave-ss4n krb5kdc[21647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045048, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:37:28.969830 22358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:28.914114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:36365 (local address 127.10.25.62:36075)
0806 19:37:28.931063 (+ 16949us) server_negotiation.cc:206] Beginning negotiation
0806 19:37:28.931074 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:37:28.931111 (+ 37us) server_negotiation.cc:407] Connection header received
0806 19:37:28.931211 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:37:28.931221 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:37:28.931346 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:37:28.931536 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:37:28.949945 (+ 18409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:28.951295 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:28.953287 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:28.953839 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:28.954000 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:28.959704 (+ 5704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:37:28.959712 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:37:28.959717 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:37:28.959767 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:37:28.963699 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:37:28.965865 (+ 2166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:37:28.965877 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:37:28.965881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:37:28.966135 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:37:28.967141 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:37:28.967147 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:37:28.967150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:37:28.967612 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:37:28.967754 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 19:37:28.968998 (+ 1244us) server_negotiation.cc:299] Negotiation successful
0806 19:37:28.969311 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16724,"thread_start_us":17595,"threads_started":1}
I20260806 19:37:28.973922 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:28.913242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36075 (local address 127.10.25.1:36365)
0806 19:37:28.925030 (+ 11788us) negotiation.cc:107] Waiting for socket to connect
0806 19:37:28.925108 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 19:37:28.928692 (+ 3584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:37:28.947376 (+ 18684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:37:28.947393 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:37:28.948393 (+ 1000us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:37:28.949632 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:37:28.949668 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:37:28.951558 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:37:28.951562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:37:28.953083 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:37:28.953100 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:37:28.954134 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:37:28.954142 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:37:28.954476 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:28.955406 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:37:28.955440 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:37:28.959316 (+ 3876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:37:28.963984 (+ 4668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:37:28.963995 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:37:28.964019 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:37:28.965437 (+ 1418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:37:28.966412 (+ 975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:37:28.966418 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:37:28.966422 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:37:28.966935 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:37:28.967898 (+ 963us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:37:28.967908 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:37:28.968452 (+ 544us) client_negotiation.cc:769] Sending connection context
0806 19:37:28.970950 (+ 2498us) client_negotiation.cc:240] Negotiation successful
0806 19:37:28.971822 (+ 872us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11325,"thread_start_us":15481,"threads_started":1}
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
I20260806 19:37:28.976890 22355 heartbeater.cc:347] Connected to a master server at 127.10.25.62:36075
I20260806 19:37:28.977285 22355 heartbeater.cc:464] Registering TS with master...
I20260806 19:37:28.978492 22355 heartbeater.cc:511] Master 127.10.25.62:36075 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
I20260806 19:37:28.981554 22137 ts_manager.cc:194] Registered new tserver with Master: d1a9aa6111454428bd610813e0da907c (127.10.25.1:33621)
I20260806 19:37:28.984215 22137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:36365
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:37:29.030089 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.25.20:39867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36075
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:37:29 dist-test-slave-ss4n krb5kdc[21647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045048, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
W20260806 19:37:29.643788 22361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:37:29.644415 22361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:37:29.644675 22361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:37:29.658639 22361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:37:29.658933 22361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:37:29.659216 22361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:37:29.677235 22361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.25.20:39867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:37:29.679445 22361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:37:29.681864 22361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:37:29.702266 22375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:37:29.703563 22376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:37:29.705330 22379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:37:29.706418 22361 server_base.cc:1034] running on GCE node
I20260806 19:37:29.707273 22361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:37:29.708633 22361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:37:29.709949 22361 hybrid_clock.cc:648] HybridClock initialized: now 1786045049709871 us; error 66 us; skew 500 ppm
I20260806 19:37:29.710369 22361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:37:29 dist-test-slave-ss4n krb5kdc[21647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045049, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:37:29.715735 22361 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:37:29.720351 22361 webserver.cc:460] Webserver started at http://127.10.25.2:46319/ using document root <none> and password file <none>
I20260806 19:37:29.721731 22361 fs_manager.cc:359] Metadata directory not provided
I20260806 19:37:29.721927 22361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:37:29.722546 22361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 19:37:29 dist-test-slave-ss4n krb5kdc[21647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045049, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
I20260806 19:37:29.744359 22361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "7af2b8ed860141409bda2f785dacd40d"
format_stamp: "Formatted at 2026-08-06 19:37:29 on dist-test-slave-ss4n"
server_key: "43d6bb813f049e017e365f1eead287de"
server_key_iv: "8415216affec22a5c100ef48165b716d"
server_key_version: "kuduclusterkey@0"
I20260806 19:37:29.746955 22361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "7af2b8ed860141409bda2f785dacd40d"
format_stamp: "Formatted at 2026-08-06 19:37:29 on dist-test-slave-ss4n"
server_key: "43d6bb813f049e017e365f1eead287de"
server_key_iv: "8415216affec22a5c100ef48165b716d"
server_key_version: "kuduclusterkey@0"
I20260806 19:37:29.755755 22361 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260806 19:37:29.782768 22386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:37:29.785172 22361 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.004s
I20260806 19:37:29.785441 22361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "7af2b8ed860141409bda2f785dacd40d"
format_stamp: "Formatted at 2026-08-06 19:37:29 on dist-test-slave-ss4n"
server_key: "43d6bb813f049e017e365f1eead287de"
server_key_iv: "8415216affec22a5c100ef48165b716d"
server_key_version: "kuduclusterkey@0"
I20260806 19:37:29.785844 22361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:37:29.809778 22361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:37:29.816042 22361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:37:29.816421 22361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:37:29.817858 22361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:37:29.820413 22361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:37:29.820533 22361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:37:29.820674 22361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:37:29.820748 22361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:37:29.887575 22361 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:33913
I20260806 19:37:29.887674 22500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:33913 every 8 connection(s)
I20260806 19:37:29.889681 22361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:37:29.897644 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 22361
I20260806 19:37:29.897859 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:37:29.898502 10340 external_mini_cluster.cc:1428] Setting key 69fc91ab152eb42b541c7534c0f8adf4
Aug 06 19:37:29 dist-test-slave-ss4n krb5kdc[21647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045049, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:37:29.922765 22503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:29.893274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:56303 (local address 127.10.25.62:36075)
0806 19:37:29.894034 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0806 19:37:29.894041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:37:29.896125 (+ 2084us) server_negotiation.cc:407] Connection header received
0806 19:37:29.899218 (+ 3093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:37:29.899227 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:37:29.899336 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:37:29.899427 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:37:29.902462 (+ 3035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:29.903642 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:29.906341 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:29.906992 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:29.907095 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:29.912410 (+ 5315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:37:29.912421 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:37:29.912427 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:37:29.912497 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:37:29.917144 (+ 4647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:37:29.918972 (+ 1828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:37:29.918982 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:37:29.918986 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:37:29.919227 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:37:29.919927 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:37:29.919934 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:37:29.919944 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:37:29.920378 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:37:29.920604 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 19:37:29.921758 (+ 1154us) server_negotiation.cc:299] Negotiation successful
0806 19:37:29.922084 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":452,"threads_started":1}
I20260806 19:37:29.925359 22504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:29.894546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36075 (local address 127.10.25.2:56303)
0806 19:37:29.895925 (+ 1379us) negotiation.cc:107] Waiting for socket to connect
0806 19:37:29.895981 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 19:37:29.898797 (+ 2816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:37:29.900310 (+ 1513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:37:29.900323 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:37:29.901214 (+ 891us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:37:29.902221 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:37:29.902251 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:37:29.903875 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:37:29.903881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:37:29.906104 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:37:29.906133 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:37:29.907500 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:37:29.907507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:37:29.907832 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:29.908706 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:37:29.908736 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:37:29.912020 (+ 3284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:37:29.917637 (+ 5617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:37:29.917653 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:37:29.917685 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:37:29.918713 (+ 1028us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:37:29.919434 (+ 721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:37:29.919440 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:37:29.919445 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:37:29.919772 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:37:29.920721 (+ 949us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:37:29.920731 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:37:29.921221 (+ 490us) client_negotiation.cc:769] Sending connection context
0806 19:37:29.923080 (+ 1859us) client_negotiation.cc:240] Negotiation successful
0806 19:37:29.923786 (+ 706us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1019,"thread_start_us":514,"threads_started":1}
I20260806 19:37:29.928010 22501 heartbeater.cc:347] Connected to a master server at 127.10.25.62:36075
I20260806 19:37:29.928390 22501 heartbeater.cc:464] Registering TS with master...
I20260806 19:37:29.929365 22501 heartbeater.cc:511] Master 127.10.25.62:36075 requested a full tablet report, sending...
I20260806 19:37:29.931547 22137 ts_manager.cc:194] Registered new tserver with Master: 7af2b8ed860141409bda2f785dacd40d (127.10.25.2:33913)
I20260806 19:37:29.932797 22137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:56303
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:37:29.991050 22355 heartbeater.cc:503] Master 127.10.25.62:36075 was elected leader, sending a full tablet report...
I20260806 19:37:29.991170 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.25.20:39867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36075
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:37:30.500926 22507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:37:30.501272 22507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:37:30.501456 22507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:37:30.510541 22507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:37:30.510737 22507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:37:30.510919 22507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:37:30.523721 22507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.25.20:39867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:37:30.525343 22507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:37:30.526844 22507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:37:30.545828 22512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:37:30.546051 22513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:37:30.546092 22515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:37:30.547125 22507 server_base.cc:1034] running on GCE node
I20260806 19:37:30.547959 22507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:37:30.549285 22507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:37:30.550678 22507 hybrid_clock.cc:648] HybridClock initialized: now 1786045050550632 us; error 55 us; skew 500 ppm
I20260806 19:37:30.551177 22507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:37:30 dist-test-slave-ss4n krb5kdc[21647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045050, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:37:30.555030 22507 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:37:30.556888 22507 webserver.cc:460] Webserver started at http://127.10.25.3:34683/ using document root <none> and password file <none>
I20260806 19:37:30.557730 22507 fs_manager.cc:359] Metadata directory not provided
I20260806 19:37:30.557855 22507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:37:30.558147 22507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 19:37:30 dist-test-slave-ss4n krb5kdc[21647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045050, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
I20260806 19:37:30.576139 22507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "fd30954c77944091826657740f56a4bf"
format_stamp: "Formatted at 2026-08-06 19:37:30 on dist-test-slave-ss4n"
server_key: "223a49db14d96b61a0c5cbf1ba867925"
server_key_iv: "357749af1461c9dd7cb4c621bb0fabe6"
server_key_version: "kuduclusterkey@0"
I20260806 19:37:30.577247 22507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "fd30954c77944091826657740f56a4bf"
format_stamp: "Formatted at 2026-08-06 19:37:30 on dist-test-slave-ss4n"
server_key: "223a49db14d96b61a0c5cbf1ba867925"
server_key_iv: "357749af1461c9dd7cb4c621bb0fabe6"
server_key_version: "kuduclusterkey@0"
I20260806 19:37:30.585620 22507 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.001s sys 0.005s
I20260806 19:37:30.602924 22522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:37:30.605123 22507 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 19:37:30.605355 22507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "fd30954c77944091826657740f56a4bf"
format_stamp: "Formatted at 2026-08-06 19:37:30 on dist-test-slave-ss4n"
server_key: "223a49db14d96b61a0c5cbf1ba867925"
server_key_iv: "357749af1461c9dd7cb4c621bb0fabe6"
server_key_version: "kuduclusterkey@0"
I20260806 19:37:30.605691 22507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:37:30.633234 22507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:37:30.639037 22507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:37:30.639422 22507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:37:30.640651 22507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:37:30.642354 22507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:37:30.642472 22507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:37:30.642642 22507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:37:30.642717 22507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:37:30.689038 22507 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:42779
I20260806 19:37:30.689066 22636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:42779 every 8 connection(s)
I20260806 19:37:30.690562 22507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:37:30.695211 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 22507
I20260806 19:37:30.695437 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:37:30.696080 10340 external_mini_cluster.cc:1428] Setting key 081063f13ef3414b8aefe1db90ac530f
Aug 06 19:37:30 dist-test-slave-ss4n krb5kdc[21647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045050, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:37:30.717979 22639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:30.694152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:52987 (local address 127.10.25.62:36075)
0806 19:37:30.694669 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0806 19:37:30.694674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:37:30.696027 (+ 1353us) server_negotiation.cc:407] Connection header received
0806 19:37:30.699278 (+ 3251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:37:30.699285 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:37:30.699454 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:37:30.699722 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:37:30.701855 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.702634 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.704488 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.704957 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.705114 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:30.710045 (+ 4931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:37:30.710052 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:37:30.710056 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:37:30.710106 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:37:30.714183 (+ 4077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:37:30.715129 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:37:30.715136 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:37:30.715140 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:37:30.715301 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:37:30.715874 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:37:30.715880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:37:30.715884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:37:30.716214 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:37:30.716410 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 19:37:30.717217 (+ 807us) server_negotiation.cc:299] Negotiation successful
0806 19:37:30.717520 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":288,"threads_started":1}
I20260806 19:37:30.719125 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:30.694620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36075 (local address 127.10.25.3:52987)
0806 19:37:30.695823 (+ 1203us) negotiation.cc:107] Waiting for socket to connect
0806 19:37:30.695873 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 19:37:30.698751 (+ 2878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:37:30.700194 (+ 1443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:37:30.700206 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:37:30.700747 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:37:30.701587 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:37:30.701623 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.702841 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.702846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:37:30.704268 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:37:30.704292 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.705241 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.705249 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:37:30.705583 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:30.706459 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:37:30.706502 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:37:30.709647 (+ 3145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:37:30.714370 (+ 4723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:37:30.714382 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:37:30.714407 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:37:30.714970 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:37:30.715457 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:37:30.715463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:37:30.715468 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:37:30.715732 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:37:30.716371 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:37:30.716381 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:37:30.716800 (+ 419us) client_negotiation.cc:769] Sending connection context
0806 19:37:30.717112 (+ 312us) client_negotiation.cc:240] Negotiation successful
0806 19:37:30.717809 (+ 697us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"thread_start_us":545,"threads_started":1}
I20260806 19:37:30.721216 22637 heartbeater.cc:347] Connected to a master server at 127.10.25.62:36075
I20260806 19:37:30.721611 22637 heartbeater.cc:464] Registering TS with master...
I20260806 19:37:30.722440 22637 heartbeater.cc:511] Master 127.10.25.62:36075 requested a full tablet report, sending...
I20260806 19:37:30.724360 22137 ts_manager.cc:194] Registered new tserver with Master: fd30954c77944091826657740f56a4bf (127.10.25.3:42779)
I20260806 19:37:30.725317 22137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:52987
I20260806 19:37:30.726083 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:37:30.752470 22639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:30.732861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41614 (local address 127.10.25.62:36075)
0806 19:37:30.733203 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0806 19:37:30.733210 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:37:30.733798 (+ 588us) server_negotiation.cc:407] Connection header received
0806 19:37:30.733978 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:37:30.733982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:37:30.734087 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:37:30.734194 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:37:30.736200 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.737162 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.739770 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.740281 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.740405 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:30.744649 (+ 4244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:37:30.744658 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:37:30.744663 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:37:30.744732 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:37:30.748955 (+ 4223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:37:30.750145 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:37:30.750152 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:37:30.750156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:37:30.750336 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:37:30.750808 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:37:30.750812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:37:30.750814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:37:30.751137 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:37:30.751343 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 19:37:30.751980 (+ 637us) server_negotiation.cc:299] Negotiation successful
0806 19:37:30.752220 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260806 19:37:30.759346 22137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:37:30.768805 22137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:37:30.812107 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:30.789869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45844 (local address 127.10.25.1:33621)
0806 19:37:30.802511 (+ 12642us) server_negotiation.cc:206] Beginning negotiation
0806 19:37:30.802523 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:37:30.802556 (+ 33us) server_negotiation.cc:407] Connection header received
0806 19:37:30.802920 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:37:30.802931 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:37:30.803234 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:37:30.803626 (+ 392us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:37:30.804419 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.806293 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.808789 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.810468 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.810624 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:30.810908 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0806 19:37:30.811514 (+ 606us) server_negotiation.cc:299] Negotiation successful
0806 19:37:30.811713 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12459,"thread_start_us":12171,"threads_started":1}
I20260806 19:37:30.812147 22651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:30.788169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52104 (local address 127.10.25.3:42779)
0806 19:37:30.800713 (+ 12544us) server_negotiation.cc:206] Beginning negotiation
0806 19:37:30.800729 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:37:30.800786 (+ 57us) server_negotiation.cc:407] Connection header received
0806 19:37:30.800940 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:37:30.800953 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:37:30.801335 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:37:30.802255 (+ 920us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:37:30.802891 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.804709 (+ 1818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.808517 (+ 3808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.810648 (+ 2131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.811102 (+ 454us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:30.811273 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 19:37:30.811518 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 19:37:30.811707 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12240,"thread_start_us":13564,"threads_started":1}
I20260806 19:37:30.812171 22650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:30.787712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:42779 (local address 127.0.0.1:52104)
0806 19:37:30.788686 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0806 19:37:30.788737 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 19:37:30.789109 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:37:30.802047 (+ 12938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:37:30.802055 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:37:30.802120 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:37:30.802683 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:37:30.802694 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.805386 (+ 2692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.805393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:37:30.808269 (+ 2876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:37:30.808291 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.810930 (+ 2639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.810938 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:37:30.811206 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:30.811286 (+ 80us) client_negotiation.cc:769] Sending connection context
0806 19:37:30.811435 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 19:37:30.811687 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":46,"spinlock_wait_cycles":1408,"thread_start_us":466,"threads_started":1}
I20260806 19:37:30.812315 22652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:30.788416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:33913 (local address 127.0.0.1:45416)
0806 19:37:30.799833 (+ 11417us) negotiation.cc:107] Waiting for socket to connect
0806 19:37:30.799869 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 19:37:30.800307 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:37:30.802039 (+ 1732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:37:30.802048 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:37:30.802100 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:37:30.803096 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:37:30.803105 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.804909 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.804918 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:37:30.808269 (+ 3351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:37:30.808291 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.810853 (+ 2562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.810862 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:37:30.811634 (+ 772us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:30.811663 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 19:37:30.811748 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 19:37:30.811888 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11190,"mutex_wait_us":31,"thread_start_us":12040,"threads_started":1}
I20260806 19:37:30.813696 22653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:30.789126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45416 (local address 127.10.25.2:33913)
0806 19:37:30.801180 (+ 12054us) server_negotiation.cc:206] Beginning negotiation
0806 19:37:30.801192 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:37:30.801228 (+ 36us) server_negotiation.cc:407] Connection header received
0806 19:37:30.801339 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:37:30.801347 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:37:30.801613 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:37:30.802593 (+ 980us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:37:30.803272 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.804726 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.808526 (+ 3800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.810612 (+ 2086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.810796 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:30.811025 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 19:37:30.812512 (+ 1487us) server_negotiation.cc:299] Negotiation successful
0806 19:37:30.813328 (+ 816us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11878,"thread_start_us":11760,"threads_started":1}
I20260806 19:37:30.813757 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:30.790315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:33621 (local address 127.0.0.1:45844)
0806 19:37:30.801573 (+ 11258us) negotiation.cc:107] Waiting for socket to connect
0806 19:37:30.801604 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 19:37:30.802687 (+ 1083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:37:30.803623 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:37:30.803628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:37:30.803653 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:37:30.804032 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:37:30.804042 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.806508 (+ 2466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.806512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:37:30.808401 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:37:30.808418 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.811071 (+ 2653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.811078 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:37:30.811258 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:30.811285 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 19:37:30.811435 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 19:37:30.813013 (+ 1578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11103,"mutex_wait_us":49,"thread_start_us":10969,"threads_started":1}
I20260806 19:37:30.817726 22289 tablet_service.cc:1467] Processing CreateTablet for tablet 3e8a22cd5b5e42e2ae1c5570c2870bbe (DEFAULT_TABLE table=test-table [id=1a0f7475b4294ffa831dd267832df7cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:37:30.817799 22571 tablet_service.cc:1467] Processing CreateTablet for tablet 3e8a22cd5b5e42e2ae1c5570c2870bbe (DEFAULT_TABLE table=test-table [id=1a0f7475b4294ffa831dd267832df7cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:37:30.817790 22434 tablet_service.cc:1467] Processing CreateTablet for tablet 3e8a22cd5b5e42e2ae1c5570c2870bbe (DEFAULT_TABLE table=test-table [id=1a0f7475b4294ffa831dd267832df7cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:37:30.819272 22434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e8a22cd5b5e42e2ae1c5570c2870bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:37:30.819275 22289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e8a22cd5b5e42e2ae1c5570c2870bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:37:30.819554 22571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e8a22cd5b5e42e2ae1c5570c2870bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:37:30.836066 22658 tablet_bootstrap.cc:492] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c: Bootstrap starting.
I20260806 19:37:30.838601 22657 tablet_bootstrap.cc:492] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf: Bootstrap starting.
I20260806 19:37:30.841766 22656 tablet_bootstrap.cc:492] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d: Bootstrap starting.
I20260806 19:37:30.842052 22658 tablet_bootstrap.cc:654] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c: Neither blocks nor log segments found. Creating new log.
I20260806 19:37:30.843737 22658 log.cc:824] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c: Log is configured to *not* fsync() on all Append() calls
I20260806 19:37:30.844323 22657 tablet_bootstrap.cc:654] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf: Neither blocks nor log segments found. Creating new log.
I20260806 19:37:30.845521 22656 tablet_bootstrap.cc:654] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d: Neither blocks nor log segments found. Creating new log.
I20260806 19:37:30.846692 22657 log.cc:824] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf: Log is configured to *not* fsync() on all Append() calls
I20260806 19:37:30.847021 22656 log.cc:824] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d: Log is configured to *not* fsync() on all Append() calls
I20260806 19:37:30.848874 22658 tablet_bootstrap.cc:492] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c: No bootstrap required, opened a new log
I20260806 19:37:30.849269 22658 ts_tablet_manager.cc:1397] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260806 19:37:30.851362 22656 tablet_bootstrap.cc:492] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d: No bootstrap required, opened a new log
I20260806 19:37:30.851408 22657 tablet_bootstrap.cc:492] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf: No bootstrap required, opened a new log
I20260806 19:37:30.851701 22656 ts_tablet_manager.cc:1397] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260806 19:37:30.851730 22657 ts_tablet_manager.cc:1397] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260806 19:37:30.863382 22657 raft_consensus.cc:348] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd30954c77944091826657740f56a4bf" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 42779 } } peers { permanent_uuid: "d1a9aa6111454428bd610813e0da907c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 33621 } } peers { permanent_uuid: "7af2b8ed860141409bda2f785dacd40d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33913 } }
I20260806 19:37:30.863929 22657 raft_consensus.cc:374] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:37:30.863411 22658 raft_consensus.cc:348] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd30954c77944091826657740f56a4bf" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 42779 } } peers { permanent_uuid: "d1a9aa6111454428bd610813e0da907c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 33621 } } peers { permanent_uuid: "7af2b8ed860141409bda2f785dacd40d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33913 } }
I20260806 19:37:30.864112 22658 raft_consensus.cc:374] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:37:30.864156 22657 raft_consensus.cc:729] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd30954c77944091826657740f56a4bf, State: Initialized, Role: FOLLOWER
I20260806 19:37:30.864240 22658 raft_consensus.cc:729] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1a9aa6111454428bd610813e0da907c, State: Initialized, Role: FOLLOWER
I20260806 19:37:30.865034 22658 consensus_queue.cc:260] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd30954c77944091826657740f56a4bf" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 42779 } } peers { permanent_uuid: "d1a9aa6111454428bd610813e0da907c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 33621 } } peers { permanent_uuid: "7af2b8ed860141409bda2f785dacd40d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33913 } }
I20260806 19:37:30.865083 22657 consensus_queue.cc:260] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd30954c77944091826657740f56a4bf" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 42779 } } peers { permanent_uuid: "d1a9aa6111454428bd610813e0da907c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 33621 } } peers { permanent_uuid: "7af2b8ed860141409bda2f785dacd40d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33913 } }
I20260806 19:37:30.865744 22656 raft_consensus.cc:348] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd30954c77944091826657740f56a4bf" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 42779 } } peers { permanent_uuid: "d1a9aa6111454428bd610813e0da907c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 33621 } } peers { permanent_uuid: "7af2b8ed860141409bda2f785dacd40d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33913 } }
I20260806 19:37:30.866380 22656 raft_consensus.cc:374] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:37:30.866516 22656 raft_consensus.cc:729] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af2b8ed860141409bda2f785dacd40d, State: Initialized, Role: FOLLOWER
I20260806 19:37:30.867333 22656 consensus_queue.cc:260] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd30954c77944091826657740f56a4bf" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 42779 } } peers { permanent_uuid: "d1a9aa6111454428bd610813e0da907c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 33621 } } peers { permanent_uuid: "7af2b8ed860141409bda2f785dacd40d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33913 } }
I20260806 19:37:30.867769 22658 ts_tablet_manager.cc:1428] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c: Time spent starting tablet: real 0.018s user 0.019s sys 0.000s
I20260806 19:37:30.868083 22637 heartbeater.cc:503] Master 127.10.25.62:36075 was elected leader, sending a full tablet report...
I20260806 19:37:30.869809 22657 ts_tablet_manager.cc:1428] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf: Time spent starting tablet: real 0.018s user 0.018s sys 0.000s
I20260806 19:37:30.871089 22501 heartbeater.cc:503] Master 127.10.25.62:36075 was elected leader, sending a full tablet report...
I20260806 19:37:30.872092 22656 ts_tablet_manager.cc:1428] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d: Time spent starting tablet: real 0.020s user 0.017s sys 0.001s
W20260806 19:37:30.892803 22502 tablet.cc:2369] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:37:30.942745 22638 tablet.cc:2369] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:37:30.948385 22356 tablet.cc:2369] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:37:30.952631 22663 raft_consensus.cc:482] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:37:30.953022 22663 raft_consensus.cc:504] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd30954c77944091826657740f56a4bf" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 42779 } } peers { permanent_uuid: "d1a9aa6111454428bd610813e0da907c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 33621 } } peers { permanent_uuid: "7af2b8ed860141409bda2f785dacd40d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33913 } }
I20260806 19:37:30.955168 22663 leader_election.cc:288] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1a9aa6111454428bd610813e0da907c (127.10.25.1:33621), 7af2b8ed860141409bda2f785dacd40d (127.10.25.2:33913)
I20260806 19:37:30.962150 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:30.955526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:33913 (local address 127.10.25.3:37649)
0806 19:37:30.955784 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 19:37:30.955808 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:37:30.955960 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:37:30.956683 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:37:30.956691 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:37:30.956728 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:37:30.957091 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:37:30.957109 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.958280 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.958286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:37:30.959649 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:37:30.959662 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.961078 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.961084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:37:30.961358 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:30.961400 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 19:37:30.961577 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 19:37:30.961747 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":1}
I20260806 19:37:30.962568 22653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:30.955854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:37649 (local address 127.10.25.2:33913)
0806 19:37:30.956239 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0806 19:37:30.956248 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:37:30.956271 (+ 23us) server_negotiation.cc:407] Connection header received
0806 19:37:30.956354 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:37:30.956360 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:37:30.956458 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:37:30.956540 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:37:30.957291 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.958134 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.959834 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.960803 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.961049 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:30.961156 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 19:37:30.961771 (+ 615us) server_negotiation.cc:299] Negotiation successful
0806 19:37:30.962014 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260806 19:37:30.964480 22454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e8a22cd5b5e42e2ae1c5570c2870bbe" candidate_uuid: "fd30954c77944091826657740f56a4bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af2b8ed860141409bda2f785dacd40d" is_pre_election: true
I20260806 19:37:30.964972 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:30.955737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:60487 (local address 127.10.25.1:33621)
0806 19:37:30.955986 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 19:37:30.955995 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:37:30.956736 (+ 741us) server_negotiation.cc:407] Connection header received
0806 19:37:30.957201 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:37:30.957208 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:37:30.957593 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:37:30.957732 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:37:30.958464 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.959336 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.961980 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.963690 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.963882 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:30.964041 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 19:37:30.964407 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 19:37:30.964570 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 19:37:30.965121 22665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:30.955526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:33621 (local address 127.10.25.3:60487)
0806 19:37:30.956427 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0806 19:37:30.956442 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 19:37:30.956852 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:37:30.957749 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:37:30.957754 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:37:30.957783 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:37:30.958273 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:37:30.958284 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.959529 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.959534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:37:30.961742 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:37:30.961761 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:37:30.963941 (+ 2180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:37:30.963947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:37:30.964176 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:30.964205 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 19:37:30.964332 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 19:37:30.964493 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":639,"spinlock_wait_cycles":39680,"thread_start_us":419,"threads_started":1}
I20260806 19:37:30.965643 22454 raft_consensus.cc:2454] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd30954c77944091826657740f56a4bf in term 0.
I20260806 19:37:30.966766 22309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e8a22cd5b5e42e2ae1c5570c2870bbe" candidate_uuid: "fd30954c77944091826657740f56a4bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1a9aa6111454428bd610813e0da907c" is_pre_election: true
I20260806 19:37:30.967077 22526 leader_election.cc:302] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7af2b8ed860141409bda2f785dacd40d, fd30954c77944091826657740f56a4bf; no voters:
I20260806 19:37:30.967545 22309 raft_consensus.cc:2454] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd30954c77944091826657740f56a4bf in term 0.
I20260806 19:37:30.967733 22663 raft_consensus.cc:2781] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:37:30.967976 22663 raft_consensus.cc:482] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:37:30.968163 22663 raft_consensus.cc:3037] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:37:30.971762 22663 raft_consensus.cc:504] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd30954c77944091826657740f56a4bf" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 42779 } } peers { permanent_uuid: "d1a9aa6111454428bd610813e0da907c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 33621 } } peers { permanent_uuid: "7af2b8ed860141409bda2f785dacd40d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33913 } }
I20260806 19:37:30.972929 22663 leader_election.cc:288] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [CANDIDATE]: Term 1 election: Requested vote from peers d1a9aa6111454428bd610813e0da907c (127.10.25.1:33621), 7af2b8ed860141409bda2f785dacd40d (127.10.25.2:33913)
I20260806 19:37:30.974112 22454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e8a22cd5b5e42e2ae1c5570c2870bbe" candidate_uuid: "fd30954c77944091826657740f56a4bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af2b8ed860141409bda2f785dacd40d"
I20260806 19:37:30.974116 22309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e8a22cd5b5e42e2ae1c5570c2870bbe" candidate_uuid: "fd30954c77944091826657740f56a4bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1a9aa6111454428bd610813e0da907c"
I20260806 19:37:30.974445 22454 raft_consensus.cc:3037] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:37:30.974469 22309 raft_consensus.cc:3037] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:37:30.978345 22309 raft_consensus.cc:2454] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd30954c77944091826657740f56a4bf in term 1.
I20260806 19:37:30.978329 22454 raft_consensus.cc:2454] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd30954c77944091826657740f56a4bf in term 1.
I20260806 19:37:30.979084 22523 leader_election.cc:302] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1a9aa6111454428bd610813e0da907c, fd30954c77944091826657740f56a4bf; no voters:
I20260806 19:37:30.979557 22663 raft_consensus.cc:2781] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:37:30.980473 22663 raft_consensus.cc:686] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [term 1 LEADER]: Becoming Leader. State: Replica: fd30954c77944091826657740f56a4bf, State: Running, Role: LEADER
I20260806 19:37:30.981135 22663 consensus_queue.cc:237] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd30954c77944091826657740f56a4bf" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 42779 } } peers { permanent_uuid: "d1a9aa6111454428bd610813e0da907c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 33621 } } peers { permanent_uuid: "7af2b8ed860141409bda2f785dacd40d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33913 } }
I20260806 19:37:30.988998 22137 catalog_manager.cc:5495] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf reported cstate change: term changed from 0 to 1, leader changed from <none> to fd30954c77944091826657740f56a4bf (127.10.25.3). New cstate: current_term: 1 leader_uuid: "fd30954c77944091826657740f56a4bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd30954c77944091826657740f56a4bf" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 42779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1a9aa6111454428bd610813e0da907c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 33621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af2b8ed860141409bda2f785dacd40d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33913 } health_report { overall_health: UNKNOWN } } }
I20260806 19:37:31.019282 22651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:31.012621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52118 (local address 127.10.25.3:42779)
0806 19:37:31.012858 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 19:37:31.012862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:37:31.013031 (+ 169us) server_negotiation.cc:407] Connection header received
0806 19:37:31.013345 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:37:31.013354 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:37:31.013616 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:37:31.013779 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:37:31.014300 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:31.015224 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:31.016794 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:31.017277 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:31.017396 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:31.018091 (+ 695us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:37:31.018679 (+ 588us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:37:31.018806 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 19:37:31.018960 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 19:37:31.019039 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 19:37:31.031922 22309 raft_consensus.cc:1261] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c [term 1 FOLLOWER]: Refusing update from remote peer fd30954c77944091826657740f56a4bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:37:31.031934 22454 raft_consensus.cc:1261] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d [term 1 FOLLOWER]: Refusing update from remote peer fd30954c77944091826657740f56a4bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:37:31.033383 22666 consensus_queue.cc:1035] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af2b8ed860141409bda2f785dacd40d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 19:37:31.034842 22663 consensus_queue.cc:1035] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1a9aa6111454428bd610813e0da907c" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 33621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:37:31.048282 22669 mvcc.cc:205] Tried to move back new op lower bound from 7315640529012850688 to 7315640528823799808. Current Snapshot: MvccSnapshot[applied={T|T < 7315640529012850688}]
I20260806 19:37:31.067576 22136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:37:31.067974 22136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:37:31.072794 22136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bab72fb52b46492ba1bbeed5009739b1: Sending DeleteTablet for 3 replicas of tablet 3e8a22cd5b5e42e2ae1c5570c2870bbe
I20260806 19:37:31.074555 22289 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e8a22cd5b5e42e2ae1c5570c2870bbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:37:31 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:45844
I20260806 19:37:31.074535 22571 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e8a22cd5b5e42e2ae1c5570c2870bbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:37:31 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:52104
I20260806 19:37:31.074916 22434 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e8a22cd5b5e42e2ae1c5570c2870bbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:37:31 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:45416
I20260806 19:37:31.075891 22679 tablet_replica.cc:330] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c: stopping tablet replica
I20260806 19:37:31.076464 22679 raft_consensus.cc:2229] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:37:31.077039 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 22220
I20260806 19:37:31.077097 22679 raft_consensus.cc:2258] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:37:31.088555 22681 tablet_replica.cc:330] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d: stopping tablet replica
I20260806 19:37:31.088608 22679 ts_tablet_manager.cc:1905] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:37:31.088899 22680 tablet_replica.cc:330] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf: stopping tablet replica
I20260806 19:37:31.089362 22680 raft_consensus.cc:2229] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:37:31.090030 22680 raft_consensus.cc:2258] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:37:31.090792 22681 raft_consensus.cc:2229] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:37:31.091411 22681 raft_consensus.cc:2258] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:37:31.094077 22681 ts_tablet_manager.cc:1905] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:37:31.094735 22680 ts_tablet_manager.cc:1905] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 19:37:31 dist-test-slave-ss4n krb5kdc[21647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:37:31.099690 22679 ts_tablet_manager.cc:1918] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:37:31.100024 22679 log.cc:1196] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/3e8a22cd5b5e42e2ae1c5570c2870bbe
I20260806 19:37:31.100824 22679 ts_tablet_manager.cc:1939] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P d1a9aa6111454428bd610813e0da907c: Deleting consensus metadata
I20260806 19:37:31.102237 22681 ts_tablet_manager.cc:1918] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:37:31.102468 22681 log.cc:1196] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/3e8a22cd5b5e42e2ae1c5570c2870bbe
I20260806 19:37:31.103327 22681 ts_tablet_manager.cc:1939] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P 7af2b8ed860141409bda2f785dacd40d: Deleting consensus metadata
I20260806 19:37:31.103659 22120 catalog_manager.cc:4841] TS d1a9aa6111454428bd610813e0da907c (127.10.25.1:33621): tablet 3e8a22cd5b5e42e2ae1c5570c2870bbe (table test-table [id=1a0f7475b4294ffa831dd267832df7cc]) successfully deleted
I20260806 19:37:31.105353 22123 catalog_manager.cc:4841] TS 7af2b8ed860141409bda2f785dacd40d (127.10.25.2:33913): tablet 3e8a22cd5b5e42e2ae1c5570c2870bbe (table test-table [id=1a0f7475b4294ffa831dd267832df7cc]) successfully deleted
I20260806 19:37:31.106350 22680 ts_tablet_manager.cc:1918] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:37:31.106668 22680 log.cc:1196] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/3e8a22cd5b5e42e2ae1c5570c2870bbe
I20260806 19:37:31.107239 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:31.078487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45846 (local address 127.10.25.1:33621)
0806 19:37:31.078671 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 19:37:31.078678 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:37:31.081147 (+ 2469us) server_negotiation.cc:407] Connection header received
0806 19:37:31.081358 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:37:31.081367 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:37:31.081457 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:37:31.081619 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:37:31.084062 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:31.085553 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:31.088237 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:31.091144 (+ 2907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:31.091320 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:31.098292 (+ 6972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:37:31.098313 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:37:31.098335 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:37:31.098385 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:37:31.102996 (+ 4611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:37:31.104161 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:37:31.104173 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:37:31.104179 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:37:31.104370 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:37:31.104909 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:37:31.104915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:37:31.104918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:37:31.105418 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:37:31.105766 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0806 19:37:31.106493 (+ 727us) server_negotiation.cc:299] Negotiation successful
0806 19:37:31.106770 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 19:37:31.107306 22680 ts_tablet_manager.cc:1939] T 3e8a22cd5b5e42e2ae1c5570c2870bbe P fd30954c77944091826657740f56a4bf: Deleting consensus metadata
I20260806 19:37:31.108026 22350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:37:31.109841 22120 catalog_manager.cc:4841] TS fd30954c77944091826657740f56a4bf (127.10.25.3:42779): tablet 3e8a22cd5b5e42e2ae1c5570c2870bbe (table test-table [id=1a0f7475b4294ffa831dd267832df7cc]) successfully deleted
I20260806 19:37:31.204731 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 22220
I20260806 19:37:31.229141 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 22361
Aug 06 19:37:31 dist-test-slave-ss4n krb5kdc[21647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:37:31.248540 22653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:31.230283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45420 (local address 127.10.25.2:33913)
0806 19:37:31.230475 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 19:37:31.230481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:37:31.230503 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:37:31.230715 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:37:31.230720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:37:31.230814 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:37:31.230923 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:37:31.232273 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:31.233047 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:31.234532 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:31.234975 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:31.235147 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:31.240660 (+ 5513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:37:31.240690 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:37:31.240708 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:37:31.240753 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:37:31.244247 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:37:31.245315 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:37:31.245324 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:37:31.245327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:37:31.245829 (+ 502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:37:31.246384 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:37:31.246389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:37:31.246391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:37:31.246915 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:37:31.247174 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0806 19:37:31.247618 (+ 444us) server_negotiation.cc:299] Negotiation successful
0806 19:37:31.247965 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 19:37:31.249390 22496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:37:31.351276 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 22361
I20260806 19:37:31.373971 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 22507
Aug 06 19:37:31 dist-test-slave-ss4n krb5kdc[21647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:37:31.396056 22651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:37:31.374904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52122 (local address 127.10.25.3:42779)
0806 19:37:31.375108 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 19:37:31.375114 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:37:31.375139 (+ 25us) server_negotiation.cc:407] Connection header received
0806 19:37:31.375320 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:37:31.375325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:37:31.375440 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:37:31.375580 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:37:31.377267 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:31.378482 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:31.380374 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:37:31.381109 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:37:31.381288 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:37:31.387657 (+ 6369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:37:31.387680 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:37:31.387703 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:37:31.387766 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:37:31.392289 (+ 4523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:37:31.393296 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:37:31.393310 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:37:31.393314 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:37:31.393548 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:37:31.394014 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:37:31.394020 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:37:31.394024 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:37:31.394554 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:37:31.394877 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0806 19:37:31.395328 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 19:37:31.395625 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 19:37:31.396860 22632 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:37:31.528889 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 22507
I20260806 19:37:31.551885 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 22104
I20260806 19:37:31.553328 22168 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:37:31.681433 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 22104
I20260806 19:37:31.710173 10340 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 19:37:32.059068 10340 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 19:37:32.059573 10340 mini_ranger.cc:66] Stopping Ranger...
I20260806 19:37:32.418576 10340 mini_ranger.cc:68] Stopped Ranger
2026-08-06 19:37:32.419 UTC [21678] LOG: received smart shutdown request
2026-08-06 19:37:32.423 UTC [21678] LOG: background worker "logical replication launcher" (PID 21687) exited with exit code 1
2026-08-06 19:37:32.424 UTC [21682] LOG: shutting down
2026-08-06 19:37:32.640 UTC [21678] LOG: database system is shut down
2026-08-06T19:37:32Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (72244 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:37:32 dist-test-slave-ss4n krb5kdc[22695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:37:32 dist-test-slave-ss4n krb5kdc[22695](info): set up 2 sockets
Aug 06 19:37:32 dist-test-slave-ss4n krb5kdc[22695](info): commencing operation
krb5kdc: starting...
W20260806 19:37:34.054160 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:37:34 dist-test-slave-ss4n krb5kdc[22695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:37:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:37:34Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.25.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.25.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.25.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/rangeradmin_127.10.25.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.25.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/rangeradmin_127.10.25.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.25.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.25.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.25.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/rangerlookup_127.10.25.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.25.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/rangerlookup_127.10.25.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.25.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.25.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.25.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/HTTP_127.10.25.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.25.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/HTTP_127.10.25.20@KRBTEST.COM.keytab.
I20260806 19:37:34.506316 10340 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskduu4iA/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/postgres -l logfile start
2026-08-06 19:37:37.970 UTC [22725] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 19:37:37.970 UTC [22725] LOG: listening on IPv4 address "127.10.25.20", port 48995
2026-08-06 19:37:37.976 UTC [22725] LOG: listening on Unix socket "/tmp/.s.PGSQL.48995"
2026-08-06 19:37:38.005 UTC [22728] LOG: database system was shut down at 2026-08-06 19:37:35 UTC
2026-08-06 19:37:38.011 UTC [22725] LOG: database system is ready to accept connections
I20260806 19:37:39.091322 10340 mini_postgres.cc:96] Postgres bound to 48995
2026-08-06 19:37:39.116 UTC [22736] FATAL: database "slave" does not exist
127.10.25.20:48995 - accepting connections
I20260806 19:37:39.116958 10340 mini_ranger.cc:161] Starting Ranger...
I20260806 19:37:39.152204 10340 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 19:37:39.826537 10340 mini_ranger.cc:87] Created ranger Postgres database
I20260806 19:37:39.826650 10340 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-admin
2026-08-06 19:37:40,172 [I] DB FLAVOR :POSTGRES
2026-08-06 19:37:40,173 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 19:37:40,173 [I] Checking connection..
2026-08-06 19:37:40,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 19:37:40,495 [I] Checking connection passed.
2026-08-06 19:37:40,495 [I] --------- Verifying version history table ---------
2026-08-06 19:37:40,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 19:37:40,819 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 19:37:40,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 19:37:41,122 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 19:37:41,122 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 19:37:41,122 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 19:37:41.412 UTC [22801] WARNING: there is no transaction in progress
2026-08-06 19:37:41,420 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 19:37:41,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 19:37:41,727 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 19:37:41,727 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 19:37:41,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 19:37:42,022 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 19:37:42,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ss4n.c.gcp-upstream.internal','N') ;"
2026-08-06 19:37:42,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 19:37:42,910 [I] Table x_portal_user does not exist in database ranger
2026-08-06 19:37:42,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 19:37:43,210 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 19:37:43,210 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 19:37:43,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 19:37:43,783 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 19:37:43,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 19:37:45.022 UTC [22910] WARNING: there is no transaction in progress
2026-08-06 19:37:45.044 UTC [22910] WARNING: there is no transaction in progress
2026-08-06 19:37:45.066 UTC [22910] WARNING: there is no transaction in progress
2026-08-06 19:37:45.087 UTC [22910] WARNING: there is no transaction in progress
2026-08-06 19:37:45.108 UTC [22910] WARNING: there is no transaction in progress
2026-08-06 19:37:45.132 UTC [22910] WARNING: there is no transaction in progress
2026-08-06 19:37:45.206 UTC [22910] WARNING: there is no transaction in progress
2026-08-06 19:37:45.215 UTC [22910] WARNING: there is no transaction in progress
2026-08-06 19:37:45.234 UTC [22910] WARNING: there is no transaction in progress
2026-08-06 19:37:45.246 UTC [22910] WARNING: there is no transaction in progress
2026-08-06 19:37:45.260 UTC [22910] WARNING: there is no transaction in progress
2026-08-06 19:37:45.280 UTC [22910] WARNING: there is no transaction in progress
2026-08-06 19:37:45.295 UTC [22910] WARNING: there is no transaction in progress
2026-08-06 19:37:45.305 UTC [22910] WARNING: there is no transaction in progress
2026-08-06 19:37:45.316 UTC [22910] WARNING: there is no transaction in progress
2026-08-06 19:37:45.326 UTC [22910] WARNING: there is no transaction in progress
2026-08-06 19:37:46,342 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 19:37:46,342 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 19:37:46,647 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 19:37:46,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 19:37:46,966 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 19:37:46,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 19:37:47,280 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 19:37:47,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 19:37:47,569 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 19:37:47,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ss4n.c.gcp-upstream.internal';"
2026-08-06 19:37:48,156 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 19:37:48,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 19:37:48,458 [I] DB_PATCHES have already been applied
I20260806 19:37:48.465974 10340 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-admin:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskduu4iA/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 19:37:48.466092 10340 mini_ranger.cc:193] Using host: 127.10.25.20
I20260806 19:37:48.487980 10340 mini_ranger.cc:227] Ranger admin URL: http://127.10.25.20:52077
Aug 06, 2026 7:37:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 7:37:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 7:37:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 7:37:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 7:37:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 7:37:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 7:37:49 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 19:37:49.793028 10340 mini_ranger.cc:160] Time spent starting Ranger: real 10.676s user 0.003s sys 0.060s
Aug 06, 2026 7:38:15 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 7:38:15 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 06, 2026 7:38:15 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 7:38:15 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 7:38:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 7:38:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 7:38:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 7:38:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 7:38:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 7:38:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 7:38:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 7:38:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 7:38:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 7:38:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 7:38:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 7:38:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 7:38:15 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 7:38:16 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 19:38:17.597390 10340 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.25.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.25.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.25.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/rangerkms_127.10.25.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.25.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/rangerkms_127.10.25.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.25.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.25.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.25.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/HTTP_127.10.25.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.25.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/HTTP_127.10.25.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 19:38:17 dist-test-slave-ss4n krb5kdc[22695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045097, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 19:38:17.828389 10340 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 19:38:17.862478 10340 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 19:38:19.016644 10340 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 19:38:19.016788 10340 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-kms
I20260806 19:38:19.016819 10340 mini_ranger_kms.cc:227] Using postgres at 127.10.25.20:48995
2026-08-06 19:38:19,450 [I] DB FLAVOR :POSTGRES
2026-08-06 19:38:19,451 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 19:38:19,451 [I] Checking connection
2026-08-06 19:38:19,451 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 19:38:19,767 [I] connection success
2026-08-06 19:38:19,767 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 19:38:19,767 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 19:38:19,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 19:38:20,103 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 19:38:20,103 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 19:38:20,103 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 19:38:20,103 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.25.20:48995/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 19:38:20,467 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 19:38:21.206938 10340 mini_ranger_kms.cc:310] Created kms service
I20260806 19:38:21.451413 10340 mini_ranger_kms.cc:326] Created kudu user
I20260806 19:38:21.521265 10340 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 19:38:21 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045097, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
I20260806 19:38:22.074360 10340 mini_ranger_kms.cc:384] Added ranger policy
I20260806 19:38:22.074728 10340 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-kms:/tmp/dist-test-taskduu4iA/build/asan/bin/postgresql.jar:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskduu4iA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 19:38:22.074779 10340 mini_ranger_kms.cc:240] Using host: 127.10.25.20
I20260806 19:38:22.103956 10340 mini_ranger_kms.cc:276] Ranger KMS PID: 23094
I20260806 19:38:22.104156 10340 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.25.20:50191
Aug 06, 2026 7:38:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 7:38:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 7:38:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 7:38:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 7:38:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 7:38:24 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 19:38:24.790926 10340 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.963s user 0.093s sys 0.375s
I20260806 19:38:24.791257 10340 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 19:38:24.791472 10340 mini_ranger_kms.cc:401] 127.10.25.20:50191/kms/v1/keys
Aug 06 19:38:25 dist-test-slave-ss4n krb5kdc[22695](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786045105, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.25.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 19:38:25 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045105, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.25.20@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
Aug 06 19:38:26 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045105, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.25.20@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
Aug 06 19:38:29 dist-test-slave-ss4n krb5kdc[22695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:38:29.107915 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:39073
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:42259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:39073
--ranger_config_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.25.20:50191/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:38:29.527251 23144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:29.527592 23144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:29.527779 23144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:29.537107 23144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:38:29.537312 23144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:29.537400 23144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:29.537537 23144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:38:29.537634 23144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:38:29.549034 23144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.25.20:50191/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:39073
--ranger_config_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:39073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:29.550557 23144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:29.552588 23144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:29.569679 23150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:29.569836 23152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:29.569836 23149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:29.572219 23144 server_base.cc:1034] running on GCE node
I20260806 19:38:29.573240 23144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:29.575687 23144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:29.576994 23144 hybrid_clock.cc:648] HybridClock initialized: now 1786045109576954 us; error 61 us; skew 500 ppm
I20260806 19:38:29.577667 23144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:29 dist-test-slave-ss4n krb5kdc[22695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045109, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:29.587095 23144 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:38:29.589399 23144 webserver.cc:460] Webserver started at http://127.10.25.62:35489/ using document root <none> and password file <none>
I20260806 19:38:29.590602 23144 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:29.590834 23144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:29.591209 23144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 19:38:29 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045109, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
I20260806 19:38:29.667487 23144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "3896fb0a982d4ac88cd529debb4e97b1"
format_stamp: "Formatted at 2026-08-06 19:38:29 on dist-test-slave-ss4n"
server_key: "e66535c4d57515464404873b4ab383c1"
server_key_iv: "4dfdee6fa45d97cd39c9c02a0fc39e45"
server_key_version: "kuduclusterkey@0"
I20260806 19:38:29.668798 23144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "3896fb0a982d4ac88cd529debb4e97b1"
format_stamp: "Formatted at 2026-08-06 19:38:29 on dist-test-slave-ss4n"
server_key: "e66535c4d57515464404873b4ab383c1"
server_key_iv: "4dfdee6fa45d97cd39c9c02a0fc39e45"
server_key_version: "kuduclusterkey@0"
I20260806 19:38:29.676450 23144 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 19:38:29.700029 23159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:29.702040 23144 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 19:38:29.702340 23144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "3896fb0a982d4ac88cd529debb4e97b1"
format_stamp: "Formatted at 2026-08-06 19:38:29 on dist-test-slave-ss4n"
server_key: "e66535c4d57515464404873b4ab383c1"
server_key_iv: "4dfdee6fa45d97cd39c9c02a0fc39e45"
server_key_version: "kuduclusterkey@0"
I20260806 19:38:29.702654 23144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:29.742538 23144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:29.748878 23144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:29.749274 23144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:29.775667 23144 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:39073
I20260806 19:38:29.775712 23210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:39073 every 8 connection(s)
I20260806 19:38:29.777356 23144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:38:29.777851 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 23144
I20260806 19:38:29.778108 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:38:29.779177 10340 external_mini_cluster.cc:1428] Setting key cc4f1feeff5f3f6c6e2ead116099a9eb
I20260806 19:38:29.783176 23211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 19:38:29 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:38:29.799916 23211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1: Bootstrap starting.
I20260806 19:38:29.804517 23211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1: Neither blocks nor log segments found. Creating new log.
I20260806 19:38:29.806020 23211 log.cc:824] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:29.808501 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:29.781719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45816 (local address 127.10.25.62:39073)
0806 19:38:29.782668 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:29.782683 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:29.782729 (+ 46us) server_negotiation.cc:407] Connection header received
0806 19:38:29.785021 (+ 2292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:29.785054 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:29.785691 (+ 637us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:29.786185 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:29.787998 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:29.790206 (+ 2208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:29.791936 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:29.792900 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:29.793169 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:29.798006 (+ 4837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:29.798021 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:29.798039 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:29.798087 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:29.802894 (+ 4807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:29.803889 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:29.803906 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:29.803914 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:29.804102 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:29.804589 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:29.804596 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:29.804599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:29.805362 (+ 763us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:29.805821 (+ 459us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:29.806353 (+ 532us) server_negotiation.cc:299] Negotiation successful
0806 19:38:29.806917 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":448,"threads_started":1}
I20260806 19:38:29.810992 23211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1: No bootstrap required, opened a new log
I20260806 19:38:29.821583 23211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3896fb0a982d4ac88cd529debb4e97b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39073 } }
I20260806 19:38:29.822261 23211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:38:29.822458 23211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3896fb0a982d4ac88cd529debb4e97b1, State: Initialized, Role: FOLLOWER
I20260806 19:38:29.823447 23211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3896fb0a982d4ac88cd529debb4e97b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39073 } }
I20260806 19:38:29.823904 23211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:38:29.824086 23211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:38:29.824326 23211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:38:29.826939 23211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3896fb0a982d4ac88cd529debb4e97b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39073 } }
I20260806 19:38:29.827621 23211 leader_election.cc:302] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3896fb0a982d4ac88cd529debb4e97b1; no voters:
I20260806 19:38:29.828553 23211 leader_election.cc:288] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:38:29.828796 23216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:38:29.830545 23216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 1 LEADER]: Becoming Leader. State: Replica: 3896fb0a982d4ac88cd529debb4e97b1, State: Running, Role: LEADER
I20260806 19:38:29.831256 23216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3896fb0a982d4ac88cd529debb4e97b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39073 } }
I20260806 19:38:29.832029 23211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:38:29.835009 23217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3896fb0a982d4ac88cd529debb4e97b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3896fb0a982d4ac88cd529debb4e97b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39073 } } }
I20260806 19:38:29.835788 23217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [sys.catalog]: This master's current role is: LEADER
I20260806 19:38:29.841364 23218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3896fb0a982d4ac88cd529debb4e97b1. Latest consensus state: current_term: 1 leader_uuid: "3896fb0a982d4ac88cd529debb4e97b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3896fb0a982d4ac88cd529debb4e97b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39073 } } }
I20260806 19:38:29.841836 23218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [sys.catalog]: This master's current role is: LEADER
I20260806 19:38:29.847558 23225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:38:29.848436 23211 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 19:38:29.860810 23225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:38:29.881932 23225 catalog_manager.cc:1347] Generated new cluster ID: 2f9e48459719461cb08b39ae222657b3
I20260806 19:38:29.882113 23225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:38:29.896612 23225 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:38:29.898794 23225 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:38:29.912135 23225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1: Generated new TSK 0
I20260806 19:38:29.913628 23225 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:38:30.913200 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.25.20:50191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39073
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:42259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:38:31 dist-test-slave-ss4n krb5kdc[22695](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786045111, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 19:38:31.454136 23257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:31.454645 23257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:31.454913 23257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:31.469538 23257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:31.469759 23257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:31.470077 23257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:38:31.490020 23257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.25.20:50191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:31.493144 23257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:31.495393 23257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:31.522352 23267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:31.528075 23268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:31.532052 23257 server_base.cc:1034] running on GCE node
W20260806 19:38:31.533056 23271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:31.534251 23257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:31.538621 23257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:31.540081 23257 hybrid_clock.cc:648] HybridClock initialized: now 1786045111539977 us; error 101 us; skew 500 ppm
I20260806 19:38:31.540663 23257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:31 dist-test-slave-ss4n krb5kdc[22695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045111, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:31.546088 23257 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:38:31.549064 23257 webserver.cc:460] Webserver started at http://127.10.25.1:40527/ using document root <none> and password file <none>
I20260806 19:38:31.550825 23257 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:31.551080 23257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:31.551620 23257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 19:38:31 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045111, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
I20260806 19:38:31.575107 23257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "ac46213f362c4733b8a698f020038d67"
format_stamp: "Formatted at 2026-08-06 19:38:31 on dist-test-slave-ss4n"
server_key: "b9f440af1ba35a795fd1c02c3c3930e5"
server_key_iv: "62e6d729995592fccab5acb7ba54ea79"
server_key_version: "kuduclusterkey@0"
I20260806 19:38:31.576493 23257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "ac46213f362c4733b8a698f020038d67"
format_stamp: "Formatted at 2026-08-06 19:38:31 on dist-test-slave-ss4n"
server_key: "b9f440af1ba35a795fd1c02c3c3930e5"
server_key_iv: "62e6d729995592fccab5acb7ba54ea79"
server_key_version: "kuduclusterkey@0"
I20260806 19:38:31.585779 23257 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.003s
I20260806 19:38:31.633348 23277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:31.637969 23257 fs_manager.cc:725] Time spent opening block manager: real 0.023s user 0.005s sys 0.005s
I20260806 19:38:31.638383 23257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "ac46213f362c4733b8a698f020038d67"
format_stamp: "Formatted at 2026-08-06 19:38:31 on dist-test-slave-ss4n"
server_key: "b9f440af1ba35a795fd1c02c3c3930e5"
server_key_iv: "62e6d729995592fccab5acb7ba54ea79"
server_key_version: "kuduclusterkey@0"
I20260806 19:38:31.638847 23257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:31.724062 23257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:31.730707 23257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:31.731403 23257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:31.744468 23257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:38:31.753618 23257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:38:31.753852 23257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:31.754081 23257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:38:31.754215 23257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 19:38:31 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045111, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
I20260806 19:38:32.134917 23257 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:42039
I20260806 19:38:32.134994 23390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:42039 every 8 connection(s)
I20260806 19:38:32.136875 23257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:38:32.146750 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 23257
I20260806 19:38:32.146996 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:38:32.147881 10340 external_mini_cluster.cc:1428] Setting key 93de6a853189705375fbea0616131acf
Aug 06 19:38:32 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045111, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
I20260806 19:38:32.198295 23393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:32.140722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:55245 (local address 127.10.25.62:39073)
0806 19:38:32.158176 (+ 17454us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:32.158182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:32.168113 (+ 9931us) server_negotiation.cc:407] Connection header received
0806 19:38:32.172214 (+ 4101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:32.172223 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:32.172394 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:32.172601 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:32.176136 (+ 3535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:32.177722 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:32.181669 (+ 3947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:32.182456 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:32.182601 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:32.189183 (+ 6582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:32.189193 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:32.189199 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:32.189261 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:32.193375 (+ 4114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:32.194641 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:32.194648 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:32.194652 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:32.194849 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:32.195458 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:32.195464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:32.195467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:32.195898 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:32.196059 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:32.197544 (+ 1485us) server_negotiation.cc:299] Negotiation successful
0806 19:38:32.197856 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17277,"thread_start_us":17027,"threads_started":1}
I20260806 19:38:32.201088 23394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:32.141119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39073 (local address 127.10.25.1:55245)
0806 19:38:32.167830 (+ 26711us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:32.167902 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:32.171663 (+ 3761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:32.173371 (+ 1708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:32.173394 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:32.174466 (+ 1072us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:32.175726 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:32.175773 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:32.178069 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:32.178076 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:32.181249 (+ 3173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:32.181290 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:32.183116 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:32.183125 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:32.183512 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:32.184568 (+ 1056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:32.184626 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:32.188478 (+ 3852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:32.193645 (+ 5167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:32.193658 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:32.193680 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:32.194411 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:32.194992 (+ 581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:32.194997 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:32.195001 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:32.195294 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:32.196086 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:32.196098 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:32.196952 (+ 854us) client_negotiation.cc:769] Sending connection context
0806 19:38:32.198896 (+ 1944us) client_negotiation.cc:240] Negotiation successful
0806 19:38:32.199569 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26248,"thread_start_us":26870,"threads_started":1}
I20260806 19:38:32.203613 23391 heartbeater.cc:347] Connected to a master server at 127.10.25.62:39073
I20260806 19:38:32.203980 23391 heartbeater.cc:464] Registering TS with master...
I20260806 19:38:32.204934 23391 heartbeater.cc:511] Master 127.10.25.62:39073 requested a full tablet report, sending...
I20260806 19:38:32.207932 23176 ts_manager.cc:194] Registered new tserver with Master: ac46213f362c4733b8a698f020038d67 (127.10.25.1:42039)
I20260806 19:38:32.210197 23176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:55245
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:38:32.244086 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.25.20:50191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39073
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:42259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:38:32 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045111, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
W20260806 19:38:32.777333 23397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:32.777799 23397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:32.777977 23397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:32.786013 23397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:32.786195 23397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:32.786391 23397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:38:32.797186 23397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.25.20:50191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:32.799108 23397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:32.800765 23397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:32.818404 23412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:32.818600 23411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:32.818568 23414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:32.819437 23397 server_base.cc:1034] running on GCE node
I20260806 19:38:32.820235 23397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:32.821379 23397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:32.822804 23397 hybrid_clock.cc:648] HybridClock initialized: now 1786045112822761 us; error 38 us; skew 500 ppm
I20260806 19:38:32.823166 23397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:32 dist-test-slave-ss4n krb5kdc[22695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045112, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:32.827078 23397 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:38:32.829358 23397 webserver.cc:460] Webserver started at http://127.10.25.2:46659/ using document root <none> and password file <none>
I20260806 19:38:32.830260 23397 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:32.830432 23397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:32.830802 23397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 19:38:32 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045112, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
I20260806 19:38:32.851226 23397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "082c1c1cab2d478ca367f57f044febf6"
format_stamp: "Formatted at 2026-08-06 19:38:32 on dist-test-slave-ss4n"
server_key: "ee7ac64e45b8da704258083f52447135"
server_key_iv: "dce851ea10af2da41de5177196f8b3da"
server_key_version: "kuduclusterkey@0"
I20260806 19:38:32.852592 23397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "082c1c1cab2d478ca367f57f044febf6"
format_stamp: "Formatted at 2026-08-06 19:38:32 on dist-test-slave-ss4n"
server_key: "ee7ac64e45b8da704258083f52447135"
server_key_iv: "dce851ea10af2da41de5177196f8b3da"
server_key_version: "kuduclusterkey@0"
I20260806 19:38:32.860859 23397 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.003s
I20260806 19:38:32.882970 23421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:32.884814 23397 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 19:38:32.885048 23397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "082c1c1cab2d478ca367f57f044febf6"
format_stamp: "Formatted at 2026-08-06 19:38:32 on dist-test-slave-ss4n"
server_key: "ee7ac64e45b8da704258083f52447135"
server_key_iv: "dce851ea10af2da41de5177196f8b3da"
server_key_version: "kuduclusterkey@0"
I20260806 19:38:32.885351 23397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:32.933876 23397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:32.938861 23397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:32.939236 23397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:32.940374 23397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:38:32.941988 23397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:38:32.942081 23397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:32.942246 23397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:38:32.942317 23397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:32.986553 23397 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:46197
I20260806 19:38:32.986608 23534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:46197 every 8 connection(s)
I20260806 19:38:32.988202 23397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:38:32.995945 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 23397
I20260806 19:38:32.996177 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:38:32.996811 10340 external_mini_cluster.cc:1428] Setting key c450ec646f92f05a68722215786e5b1f
Aug 06 19:38:32 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045112, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:38:33.014817 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:32.991489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:56803 (local address 127.10.25.62:39073)
0806 19:38:32.992089 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:32.992096 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:32.993243 (+ 1147us) server_negotiation.cc:407] Connection header received
0806 19:38:32.995410 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:32.995415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:32.995502 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:32.995614 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:32.998078 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:32.999417 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:33.001190 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:33.002070 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:33.002260 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:33.007254 (+ 4994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:33.007260 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:33.007263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:33.007321 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:33.010669 (+ 3348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:33.011995 (+ 1326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:33.012000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:33.012003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:33.012167 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:33.012761 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:33.012765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:33.012768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:33.013088 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:33.013263 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:33.014144 (+ 881us) server_negotiation.cc:299] Negotiation successful
0806 19:38:33.014401 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":283,"threads_started":1}
I20260806 19:38:33.016029 23538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:32.992102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39073 (local address 127.10.25.2:56803)
0806 19:38:32.993047 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:32.993093 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:32.995062 (+ 1969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:32.995983 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:32.996000 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:32.996719 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:32.997734 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:32.997760 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:32.999596 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:32.999602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:33.001003 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:33.001017 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:33.002374 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:33.002379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:33.002633 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:33.003472 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:33.003500 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:33.006729 (+ 3229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:33.010887 (+ 4158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:33.010899 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:33.010925 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:33.011746 (+ 821us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:33.012315 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:33.012322 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:33.012327 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:33.012614 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:33.013236 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:33.013245 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:33.013718 (+ 473us) client_negotiation.cc:769] Sending connection context
0806 19:38:33.014012 (+ 294us) client_negotiation.cc:240] Negotiation successful
0806 19:38:33.014606 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":433,"threads_started":1}
I20260806 19:38:33.017879 23535 heartbeater.cc:347] Connected to a master server at 127.10.25.62:39073
I20260806 19:38:33.018287 23535 heartbeater.cc:464] Registering TS with master...
I20260806 19:38:33.019178 23535 heartbeater.cc:511] Master 127.10.25.62:39073 requested a full tablet report, sending...
I20260806 19:38:33.021123 23176 ts_manager.cc:194] Registered new tserver with Master: 082c1c1cab2d478ca367f57f044febf6 (127.10.25.2:46197)
I20260806 19:38:33.022128 23176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:56803
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:38:33.066339 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.25.20:50191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39073
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:42259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:38:33.214735 23391 heartbeater.cc:503] Master 127.10.25.62:39073 was elected leader, sending a full tablet report...
W20260806 19:38:33.459167 23541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:33.459497 23541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:33.459606 23541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:33.467267 23541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:33.467396 23541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:33.467514 23541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:38:33.478001 23541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.25.20:50191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:33.479342 23541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:33.480700 23541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:33.496685 23546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:33.496714 23549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:33.496644 23547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:33.497752 23541 server_base.cc:1034] running on GCE node
I20260806 19:38:33.498450 23541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:33.499577 23541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:33.501057 23541 hybrid_clock.cc:648] HybridClock initialized: now 1786045113500935 us; error 63 us; skew 500 ppm
I20260806 19:38:33.501737 23541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:33 dist-test-slave-ss4n krb5kdc[22695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045113, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:33.509862 23541 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:38:33.511816 23541 webserver.cc:460] Webserver started at http://127.10.25.3:38275/ using document root <none> and password file <none>
I20260806 19:38:33.512609 23541 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:33.512823 23541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:33.513170 23541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 19:38:33 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045113, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
I20260806 19:38:33.533672 23541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "46b035327cbd491285cdd69526ed2578"
format_stamp: "Formatted at 2026-08-06 19:38:33 on dist-test-slave-ss4n"
server_key: "68ab0a3ed388b97297d07836aea1adef"
server_key_iv: "e303c8c50c22fd256163580fa1d94219"
server_key_version: "kuduclusterkey@0"
I20260806 19:38:33.534811 23541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "46b035327cbd491285cdd69526ed2578"
format_stamp: "Formatted at 2026-08-06 19:38:33 on dist-test-slave-ss4n"
server_key: "68ab0a3ed388b97297d07836aea1adef"
server_key_iv: "e303c8c50c22fd256163580fa1d94219"
server_key_version: "kuduclusterkey@0"
I20260806 19:38:33.542008 23541 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 19:38:33.562444 23556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:33.564590 23541 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 19:38:33.564828 23541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "46b035327cbd491285cdd69526ed2578"
format_stamp: "Formatted at 2026-08-06 19:38:33 on dist-test-slave-ss4n"
server_key: "68ab0a3ed388b97297d07836aea1adef"
server_key_iv: "e303c8c50c22fd256163580fa1d94219"
server_key_version: "kuduclusterkey@0"
I20260806 19:38:33.565122 23541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:33.595012 23541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:33.601150 23541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:33.601631 23541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:33.602804 23541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:38:33.604431 23541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:38:33.604563 23541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:33.604729 23541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:38:33.604812 23541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:33.655639 23668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:36431 every 8 connection(s)
I20260806 19:38:33.655853 23541 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:36431
I20260806 19:38:33.657827 23541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:38:33.662046 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 23541
I20260806 19:38:33.662290 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:38:33.663002 10340 external_mini_cluster.cc:1428] Setting key 42812014f9a29358bdfa521c848b87c5
Aug 06 19:38:33 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045113, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:38:33.689086 23672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:33.661873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:51153 (local address 127.10.25.62:39073)
0806 19:38:33.662483 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:33.662490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:33.663759 (+ 1269us) server_negotiation.cc:407] Connection header received
0806 19:38:33.666281 (+ 2522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:33.666286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:33.666385 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:33.666524 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:33.668714 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:33.670014 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:33.672889 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:33.673733 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:33.674053 (+ 320us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:33.679520 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:33.679528 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:33.679533 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:33.679589 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:33.684173 (+ 4584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:33.685628 (+ 1455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:33.685636 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:33.685642 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:33.685860 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:33.686591 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:33.686597 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:33.686600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:33.687044 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:33.687197 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:33.688392 (+ 1195us) server_negotiation.cc:299] Negotiation successful
0806 19:38:33.688701 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":261,"threads_started":1}
I20260806 19:38:33.691259 23673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:33.662559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39073 (local address 127.10.25.3:51153)
0806 19:38:33.663551 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:33.663615 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:33.665966 (+ 2351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:33.666829 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:33.666844 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:33.667467 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:33.668481 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:33.668512 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:33.670392 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:33.670399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:33.672652 (+ 2253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:33.672674 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:33.674125 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:33.674135 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:33.674513 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:33.675942 (+ 1429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:33.675970 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:33.679214 (+ 3244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:33.684403 (+ 5189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:33.684419 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:33.684446 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:33.685325 (+ 879us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:33.686050 (+ 725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:33.686058 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:33.686064 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:33.686426 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:33.687234 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:33.687246 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:33.687843 (+ 597us) client_negotiation.cc:769] Sending connection context
0806 19:38:33.688242 (+ 399us) client_negotiation.cc:240] Negotiation successful
0806 19:38:33.688992 (+ 750us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":378,"threads_started":1}
I20260806 19:38:33.693087 23669 heartbeater.cc:347] Connected to a master server at 127.10.25.62:39073
I20260806 19:38:33.693709 23669 heartbeater.cc:464] Registering TS with master...
I20260806 19:38:33.694787 23669 heartbeater.cc:511] Master 127.10.25.62:39073 requested a full tablet report, sending...
I20260806 19:38:33.697726 23175 ts_manager.cc:194] Registered new tserver with Master: 46b035327cbd491285cdd69526ed2578 (127.10.25.3:36431)
I20260806 19:38:33.699637 23175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:51153
I20260806 19:38:33.707223 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:38:33.709282 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 23257
Aug 06 19:38:33 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:38:33.733539 23675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:33.710867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49134 (local address 127.10.25.1:42039)
0806 19:38:33.711586 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:33.711594 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:33.711626 (+ 32us) server_negotiation.cc:407] Connection header received
0806 19:38:33.711696 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:33.711703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:33.711942 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:33.712201 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:33.714896 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:33.716300 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:33.718479 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:33.719798 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:33.720057 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:33.725832 (+ 5775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:33.725853 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:33.725872 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:33.725928 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:33.729458 (+ 3530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:33.730495 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:33.730504 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:33.730506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:33.730692 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:33.731177 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:33.731181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:33.731183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:33.731593 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:33.731993 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:33.732846 (+ 853us) server_negotiation.cc:299] Negotiation successful
0806 19:38:33.733094 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":390,"threads_started":1}
I20260806 19:38:33.734972 23386 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:33.824940 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 23257
I20260806 19:38:33.845216 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 23397
Aug 06 19:38:33 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:38:33.866899 23677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:33.846800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60234 (local address 127.10.25.2:46197)
0806 19:38:33.847635 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:33.847645 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:33.847685 (+ 40us) server_negotiation.cc:407] Connection header received
0806 19:38:33.847792 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:33.847804 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:33.848117 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:33.848328 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:33.850056 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:33.851754 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:33.853809 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:33.854439 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:33.854606 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:33.859893 (+ 5287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:33.859911 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:33.859929 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:33.859981 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:33.863360 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:33.864244 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:33.864255 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:33.864258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:33.864415 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:33.864797 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:33.864801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:33.864803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:33.865197 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:33.865439 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:33.866293 (+ 854us) server_negotiation.cc:299] Negotiation successful
0806 19:38:33.866562 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":487,"threads_started":1}
I20260806 19:38:33.868362 23530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:33.965868 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 23397
I20260806 19:38:33.984699 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 23541
Aug 06 19:38:33 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:38:34.006989 23679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:33.986141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53536 (local address 127.10.25.3:36431)
0806 19:38:33.986868 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:33.986875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:33.986906 (+ 31us) server_negotiation.cc:407] Connection header received
0806 19:38:33.986969 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:33.986975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:33.987224 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:33.987370 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:33.988924 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:33.990194 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:33.991933 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:33.992682 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:33.992814 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:33.998853 (+ 6039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:33.998873 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:33.998893 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:33.998959 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:34.002855 (+ 3896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:34.003868 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:34.003877 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:34.003880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:34.004070 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:34.004560 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:34.004564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:34.004566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:34.004998 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:34.005302 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:34.006286 (+ 984us) server_negotiation.cc:299] Negotiation successful
0806 19:38:34.006544 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":406,"threads_started":1}
I20260806 19:38:34.008792 23664 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:34.101787 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 23541
I20260806 19:38:34.119874 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 23144
I20260806 19:38:34.121459 23206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:34.241079 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 23144
I20260806 19:38:34.271703 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:39073
--webserver_interface=127.10.25.62
--webserver_port=35489
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:42259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:39073
--ranger_config_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.25.20:50191/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.25.20:50191/kms with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:38:34.668215 23682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:34.669029 23682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:34.669282 23682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:34.678427 23682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:38:34.678695 23682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:34.678818 23682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:34.678944 23682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:38:34.679046 23682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:38:34.692160 23682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.25.20:50191/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:39073
--ranger_config_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:39073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=35489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:34.693735 23682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:34.695204 23682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:34.714354 23687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:34.714340 23690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:34.714270 23688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:34.720564 23682 server_base.cc:1034] running on GCE node
I20260806 19:38:34.721805 23682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:34.723798 23682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:34.725200 23682 hybrid_clock.cc:648] HybridClock initialized: now 1786045114725101 us; error 100 us; skew 500 ppm
I20260806 19:38:34.726114 23682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:34 dist-test-slave-ss4n krb5kdc[22695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045114, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:34.734849 23682 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:38:34.737011 23682 webserver.cc:460] Webserver started at http://127.10.25.62:35489/ using document root <none> and password file <none>
I20260806 19:38:34.737877 23682 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:34.738013 23682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 19:38:34 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045114, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
I20260806 19:38:34.835289 23682 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 19:38:34.839985 23698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:34.842397 23682 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:38:34.842700 23682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "3896fb0a982d4ac88cd529debb4e97b1"
format_stamp: "Formatted at 2026-08-06 19:38:29 on dist-test-slave-ss4n"
server_key: "e66535c4d57515464404873b4ab383c1"
server_key_iv: "4dfdee6fa45d97cd39c9c02a0fc39e45"
server_key_version: "kuduclusterkey@0"
I20260806 19:38:34.843891 23682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:34.865461 23682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:34.870575 23682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:34.870981 23682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:34.894624 23682 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:39073
I20260806 19:38:34.894685 23751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:39073 every 8 connection(s)
I20260806 19:38:34.896425 23682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:38:34.898900 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 23682
I20260806 19:38:34.900269 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.25.20:50191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:42039
--local_ip_for_outbound_sockets=127.10.25.1
--tserver_master_addrs=127.10.25.62:39073
--webserver_port=40527
--webserver_interface=127.10.25.1
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:42259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.25.20:50191/kms with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:38:34.908800 23752 sys_catalog.cc:263] Verifying existing consensus state
I20260806 19:38:34.914426 23752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1: Bootstrap starting.
I20260806 19:38:34.938052 23752 log.cc:824] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:34.951004 23752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:38:34.951695 23752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1: Bootstrap complete.
I20260806 19:38:34.961902 23752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3896fb0a982d4ac88cd529debb4e97b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39073 } }
I20260806 19:38:34.963325 23752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3896fb0a982d4ac88cd529debb4e97b1, State: Initialized, Role: FOLLOWER
I20260806 19:38:34.964443 23752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3896fb0a982d4ac88cd529debb4e97b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39073 } }
I20260806 19:38:34.964740 23752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:38:34.964912 23752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:38:34.965157 23752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 1 FOLLOWER]: Advancing to term 2
I20260806 19:38:34.970530 23752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3896fb0a982d4ac88cd529debb4e97b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39073 } }
I20260806 19:38:34.971264 23752 leader_election.cc:302] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3896fb0a982d4ac88cd529debb4e97b1; no voters:
I20260806 19:38:34.972082 23752 leader_election.cc:288] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 19:38:34.972247 23756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 19:38:34.973951 23756 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [term 2 LEADER]: Becoming Leader. State: Replica: 3896fb0a982d4ac88cd529debb4e97b1, State: Running, Role: LEADER
I20260806 19:38:34.974956 23752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:38:34.974785 23756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3896fb0a982d4ac88cd529debb4e97b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39073 } }
I20260806 19:38:34.977535 23758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3896fb0a982d4ac88cd529debb4e97b1. Latest consensus state: current_term: 2 leader_uuid: "3896fb0a982d4ac88cd529debb4e97b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3896fb0a982d4ac88cd529debb4e97b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39073 } } }
I20260806 19:38:34.977974 23758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [sys.catalog]: This master's current role is: LEADER
I20260806 19:38:34.984125 23757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3896fb0a982d4ac88cd529debb4e97b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3896fb0a982d4ac88cd529debb4e97b1" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39073 } } }
I20260806 19:38:34.984913 23757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1 [sys.catalog]: This master's current role is: LEADER
I20260806 19:38:34.986997 23761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:38:34.995198 23761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:38:34.997464 23761 catalog_manager.cc:1259] Loaded cluster ID: 2f9e48459719461cb08b39ae222657b3
I20260806 19:38:34.997803 23761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:38:35.002825 23752 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 19:38:35.003487 23761 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:38:35.005743 23761 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1: Loaded TSK: 0
I20260806 19:38:35.008016 23761 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 19:38:35.464782 23754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:35.465351 23754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:35.465631 23754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:35.478768 23754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:35.479058 23754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:35.479329 23754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:38:35.498395 23754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.25.20:50191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:42039
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=40527
--tserver_master_addrs=127.10.25.62:39073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:35.500523 23754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:35.502713 23754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:35.545961 23791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:35.547664 23793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:35.551805 23790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:35.558197 23754 server_base.cc:1034] running on GCE node
I20260806 19:38:35.559365 23754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:35.567005 23754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:35.568570 23754 hybrid_clock.cc:648] HybridClock initialized: now 1786045115568453 us; error 90 us; skew 500 ppm
I20260806 19:38:35.569317 23754 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:35 dist-test-slave-ss4n krb5kdc[22695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045115, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:35.573884 23754 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:38:35.576790 23754 webserver.cc:460] Webserver started at http://127.10.25.1:40527/ using document root <none> and password file <none>
I20260806 19:38:35.577899 23754 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:35.578207 23754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 19:38:35 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045115, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
I20260806 19:38:35.619169 23754 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.010s sys 0.002s
I20260806 19:38:35.622973 23801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:35.625108 23754 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260806 19:38:35.625459 23754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "ac46213f362c4733b8a698f020038d67"
format_stamp: "Formatted at 2026-08-06 19:38:31 on dist-test-slave-ss4n"
server_key: "b9f440af1ba35a795fd1c02c3c3930e5"
server_key_iv: "62e6d729995592fccab5acb7ba54ea79"
server_key_version: "kuduclusterkey@0"
I20260806 19:38:35.627554 23754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:35.663916 23754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:35.672992 23754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:35.673564 23754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:35.675755 23754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:38:35.678048 23754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:38:35.678196 23754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:35.678328 23754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:38:35.678419 23754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:35.774063 23754 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:42039
I20260806 19:38:35.774115 23917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:42039 every 8 connection(s)
I20260806 19:38:35.776352 23754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:38:35.781781 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 23754
I20260806 19:38:35.782749 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.25.20:50191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:46197
--local_ip_for_outbound_sockets=127.10.25.2
--tserver_master_addrs=127.10.25.62:39073
--webserver_port=46659
--webserver_interface=127.10.25.2
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:42259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.25.20:50191/kms with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:38:35 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045115, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:38:35.828333 23921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:35.784891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:33169 (local address 127.10.25.62:39073)
0806 19:38:35.794293 (+ 9402us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:35.794313 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:35.795222 (+ 909us) server_negotiation.cc:407] Connection header received
0806 19:38:35.798082 (+ 2860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:35.798107 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:35.799955 (+ 1848us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:35.800799 (+ 844us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:35.803224 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:35.805465 (+ 2241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:35.808679 (+ 3214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:35.810066 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:35.811465 (+ 1399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:35.815776 (+ 4311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:35.815791 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:35.815819 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:35.815884 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:35.821060 (+ 5176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:35.822718 (+ 1658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:35.822732 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:35.822744 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:35.823043 (+ 299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:35.823892 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:35.823898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:35.823901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:35.824789 (+ 888us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:35.825043 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:35.826307 (+ 1264us) server_negotiation.cc:299] Negotiation successful
0806 19:38:35.826773 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9073,"thread_start_us":8758,"threads_started":1}
I20260806 19:38:35.828619 23920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:35.784803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39073 (local address 127.10.25.1:33169)
0806 19:38:35.794956 (+ 10153us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:35.795021 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:35.797134 (+ 2113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:35.801062 (+ 3928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:35.801076 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:35.801933 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:35.802863 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:35.802898 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:35.805816 (+ 2918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:35.805826 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:35.808357 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:35.808396 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:35.810406 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:35.810417 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:35.810777 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:35.811825 (+ 1048us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:35.811863 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:35.815320 (+ 3457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:35.821369 (+ 6049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:35.821385 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:35.821463 (+ 78us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:35.822431 (+ 968us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:35.823264 (+ 833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:35.823270 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:35.823274 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:35.823619 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:35.825045 (+ 1426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:35.825067 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:35.825608 (+ 541us) client_negotiation.cc:769] Sending connection context
0806 19:38:35.825992 (+ 384us) client_negotiation.cc:240] Negotiation successful
0806 19:38:35.826696 (+ 704us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9833,"thread_start_us":9608,"threads_started":1}
I20260806 19:38:35.833310 23918 heartbeater.cc:347] Connected to a master server at 127.10.25.62:39073
I20260806 19:38:35.833870 23918 heartbeater.cc:464] Registering TS with master...
I20260806 19:38:35.835053 23918 heartbeater.cc:511] Master 127.10.25.62:39073 requested a full tablet report, sending...
W20260806 19:38:35.840260 23918 heartbeater.cc:644] Failed to heartbeat to 127.10.25.62:39073 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 19:38:35.840569 23918 heartbeater.cc:464] Registering TS with master...
I20260806 19:38:35.840919 23918 heartbeater.cc:511] Master 127.10.25.62:39073 requested a full tablet report, sending...
I20260806 19:38:35.842701 23918 heartbeater.cc:464] Registering TS with master...
I20260806 19:38:35.843169 23918 heartbeater.cc:511] Master 127.10.25.62:39073 requested a full tablet report, sending...
W20260806 19:38:35.844655 23918 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 19:38:36.298094 23922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:36.298578 23922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:36.298826 23922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:36.309648 23922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:36.309783 23922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:36.309973 23922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:38:36.327898 23922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.25.20:50191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:46197
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=46659
--tserver_master_addrs=127.10.25.62:39073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:36.329936 23922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:36.331647 23922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:36.354281 23936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:36.354374 23937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:36.368566 23922 server_base.cc:1034] running on GCE node
W20260806 19:38:36.369325 23940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:36.370569 23922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:36.381964 23922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:36.385689 23922 hybrid_clock.cc:648] HybridClock initialized: now 1786045116385583 us; error 137 us; skew 500 ppm
I20260806 19:38:36.386360 23922 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:36 dist-test-slave-ss4n krb5kdc[22695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045116, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:36.391808 23922 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:38:36.396773 23922 webserver.cc:460] Webserver started at http://127.10.25.2:46659/ using document root <none> and password file <none>
I20260806 19:38:36.398238 23922 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:36.398497 23922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 19:38:36 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045116, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
I20260806 19:38:36.463950 23922 fs_manager.cc:709] Time spent opening directory manager: real 0.021s user 0.010s sys 0.007s
I20260806 19:38:36.475163 23949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:36.477626 23922 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.007s
I20260806 19:38:36.477955 23922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "082c1c1cab2d478ca367f57f044febf6"
format_stamp: "Formatted at 2026-08-06 19:38:32 on dist-test-slave-ss4n"
server_key: "ee7ac64e45b8da704258083f52447135"
server_key_iv: "dce851ea10af2da41de5177196f8b3da"
server_key_version: "kuduclusterkey@0"
I20260806 19:38:36.480024 23922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 06 19:38:36 dist-test-slave-ss4n krb5kdc[22695](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786045116, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:36.530748 23922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:36.537675 23922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:36.538147 23922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:36.545225 23922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:38:36.547732 23922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:38:36.547950 23922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 19:38:36.548116 23922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:38:36.548229 23922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:36.604946 23922 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:46197
I20260806 19:38:36.605042 24063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:46197 every 8 connection(s)
I20260806 19:38:36.607368 23922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:38:36.610854 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 23922
I20260806 19:38:36.611820 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.25.20:50191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:36431
--local_ip_for_outbound_sockets=127.10.25.3
--tserver_master_addrs=127.10.25.62:39073
--webserver_port=38275
--webserver_interface=127.10.25.3
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:42259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.25.20:50191/kms with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:38:36 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045116, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:38:36.680284 24068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:36.617941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:45911 (local address 127.10.25.62:39073)
0806 19:38:36.647136 (+ 29195us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:36.647167 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:36.648656 (+ 1489us) server_negotiation.cc:407] Connection header received
0806 19:38:36.653077 (+ 4421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:36.653086 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:36.653210 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:36.653299 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:36.656235 (+ 2936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:36.657628 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:36.660818 (+ 3190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:36.661619 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:36.661761 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:36.670155 (+ 8394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:36.670165 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:36.670171 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:36.670224 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:36.675272 (+ 5048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:36.676862 (+ 1590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:36.676870 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:36.676874 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:36.677084 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:36.677833 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:36.677839 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:36.677842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:36.678270 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:36.678432 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:36.679570 (+ 1138us) server_negotiation.cc:299] Negotiation successful
0806 19:38:36.679873 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28849,"thread_start_us":29240,"threads_started":1}
I20260806 19:38:36.683105 24067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:36.611845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39073 (local address 127.10.25.2:45911)
0806 19:38:36.648138 (+ 36293us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:36.648224 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:36.652548 (+ 4324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:36.654067 (+ 1519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:36.654081 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:36.654828 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:36.655775 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:36.655812 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:36.657929 (+ 2117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:36.657936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:36.660541 (+ 2605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:36.660581 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:36.662258 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:36.662266 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:36.662629 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:36.664780 (+ 2151us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:36.664821 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:36.669749 (+ 4928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:36.675582 (+ 5833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:36.675596 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:36.675614 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:36.676648 (+ 1034us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:36.677275 (+ 627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:36.677283 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:36.677289 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:36.677667 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:36.678575 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:36.678584 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:36.679016 (+ 432us) client_negotiation.cc:769] Sending connection context
0806 19:38:36.680851 (+ 1835us) client_negotiation.cc:240] Negotiation successful
0806 19:38:36.681641 (+ 790us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35847,"thread_start_us":35555,"threads_started":1}
I20260806 19:38:36.685137 24065 heartbeater.cc:347] Connected to a master server at 127.10.25.62:39073
I20260806 19:38:36.685616 24065 heartbeater.cc:464] Registering TS with master...
I20260806 19:38:36.686709 24065 heartbeater.cc:511] Master 127.10.25.62:39073 requested a full tablet report, sending...
I20260806 19:38:36.689370 23715 ts_manager.cc:194] Registered new tserver with Master: 082c1c1cab2d478ca367f57f044febf6 (127.10.25.2:46197)
I20260806 19:38:36.692229 23715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:45911
I20260806 19:38:36.844877 23918 heartbeater.cc:464] Registering TS with master...
I20260806 19:38:36.845438 23918 heartbeater.cc:511] Master 127.10.25.62:39073 requested a full tablet report, sending...
I20260806 19:38:36.847131 23715 ts_manager.cc:194] Registered new tserver with Master: ac46213f362c4733b8a698f020038d67 (127.10.25.1:42039)
I20260806 19:38:36.848484 23715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:33169
Aug 06 19:38:37 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045116, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
W20260806 19:38:37.165741 24069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:37.166110 24069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:37.166316 24069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:37.175657 24069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:37.175843 24069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:37.176070 24069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:38:37.189460 24069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.25.20:50191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:36431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=38275
--tserver_master_addrs=127.10.25.62:39073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:37.191227 24069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:37.192842 24069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:37.214576 24074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:37.221599 24075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:37.223934 24077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:37.228487 24069 server_base.cc:1034] running on GCE node
I20260806 19:38:37.229362 24069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:37.231824 24069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:37.233327 24069 hybrid_clock.cc:648] HybridClock initialized: now 1786045117233240 us; error 76 us; skew 500 ppm
I20260806 19:38:37.233873 24069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:37 dist-test-slave-ss4n krb5kdc[22695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045117, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:37.238971 24069 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:38:37.243533 24069 webserver.cc:460] Webserver started at http://127.10.25.3:38275/ using document root <none> and password file <none>
I20260806 19:38:37.244704 24069 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:37.244901 24069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 19:38:37 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045117, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
I20260806 19:38:37.276727 24069 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 19:38:37.281242 24085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:37.283711 24069 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 19:38:37.283977 24069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "46b035327cbd491285cdd69526ed2578"
format_stamp: "Formatted at 2026-08-06 19:38:33 on dist-test-slave-ss4n"
server_key: "68ab0a3ed388b97297d07836aea1adef"
server_key_iv: "e303c8c50c22fd256163580fa1d94219"
server_key_version: "kuduclusterkey@0"
I20260806 19:38:37.286000 24069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:37.312177 24069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:37.318380 24069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:37.318743 24069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:37.320108 24069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:38:37.321990 24069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:38:37.322114 24069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:37.322295 24069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:38:37.322373 24069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:37.405773 24069 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:36431
I20260806 19:38:37.405928 24204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:36431 every 8 connection(s)
I20260806 19:38:37.409062 24069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:38:37.418115 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 24069
Aug 06 19:38:37 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045116, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for HTTP/127.10.25.20@KRBTEST.COM
Aug 06 19:38:37 dist-test-slave-ss4n krb5kdc[22695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045117, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:38:37.470973 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:37.413311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:37557 (local address 127.10.25.62:39073)
0806 19:38:37.439681 (+ 26370us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:37.439686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:37.439711 (+ 25us) server_negotiation.cc:407] Connection header received
0806 19:38:37.441225 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:37.441232 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:37.441326 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:37.441443 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:37.443918 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.445045 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.455241 (+ 10196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.455950 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.456109 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:37.462026 (+ 5917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:37.462034 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:37.462039 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:37.462096 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:37.465841 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:37.466944 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:37.466950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:37.466954 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:37.467189 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:37.468202 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:37.468208 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:37.468212 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:37.468603 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:37.468733 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:37.470061 (+ 1328us) server_negotiation.cc:299] Negotiation successful
0806 19:38:37.470545 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26235,"spinlock_wait_cycles":13056,"thread_start_us":26081,"threads_started":1}
I20260806 19:38:37.475346 24208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:37.414106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39073 (local address 127.10.25.3:37557)
0806 19:38:37.422590 (+ 8484us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:37.422675 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:37.440704 (+ 18029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:37.442080 (+ 1376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:37.442097 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:37.442851 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:37.443700 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:37.443728 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.445270 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.445276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:37.447437 (+ 2161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:37.447459 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.456284 (+ 8825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.456296 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:37.456668 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:37.458764 (+ 2096us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:37.458800 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:37.461740 (+ 2940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:37.466049 (+ 4309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:37.466060 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:37.466082 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:37.466768 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:37.467366 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:37.467370 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:37.467373 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:37.467693 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:37.468881 (+ 1188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:37.468894 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:37.469545 (+ 651us) client_negotiation.cc:769] Sending connection context
0806 19:38:37.471598 (+ 2053us) client_negotiation.cc:240] Negotiation successful
0806 19:38:37.473738 (+ 2140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7990,"thread_start_us":7643,"threads_started":1}
I20260806 19:38:37.480558 24211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:37.439656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48092 (local address 127.10.25.62:39073)
0806 19:38:37.440255 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:37.440262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:37.440282 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:38:37.440364 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:37.440371 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:37.440485 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:37.440664 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:37.445387 (+ 4723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.446837 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.458869 (+ 12032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.459629 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.459782 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:37.466919 (+ 7137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:37.466928 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:37.466934 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:37.467042 (+ 108us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:37.475255 (+ 8213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:37.476405 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:37.476413 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:37.476417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:37.476675 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:37.477168 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:37.477174 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:37.477178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:37.477651 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:37.477989 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:37.479402 (+ 1413us) server_negotiation.cc:299] Negotiation successful
0806 19:38:37.480098 (+ 696us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"spinlock_wait_cycles":67200,"thread_start_us":734,"threads_started":1}
I20260806 19:38:37.481573 24205 heartbeater.cc:347] Connected to a master server at 127.10.25.62:39073
I20260806 19:38:37.481933 24205 heartbeater.cc:464] Registering TS with master...
I20260806 19:38:37.482982 24205 heartbeater.cc:511] Master 127.10.25.62:39073 requested a full tablet report, sending...
I20260806 19:38:37.484923 23715 ts_manager.cc:194] Registered new tserver with Master: 46b035327cbd491285cdd69526ed2578 (127.10.25.3:36431)
I20260806 19:38:37.486088 23715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:37557
I20260806 19:38:37.497004 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:38:37.534864 24211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:37.510014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48094 (local address 127.10.25.62:39073)
0806 19:38:37.510210 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:37.510216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:37.514677 (+ 4461us) server_negotiation.cc:407] Connection header received
0806 19:38:37.515095 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:37.515102 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:37.515234 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:37.515550 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:37.518562 (+ 3012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.519776 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.522709 (+ 2933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.523459 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.523597 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:37.527604 (+ 4007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:37.527612 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:37.527617 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:37.527670 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:37.531256 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:37.532330 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:37.532336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:37.532339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:37.532519 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:37.533029 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:37.533034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:37.533037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:37.533408 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:37.533680 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:37.534285 (+ 605us) server_negotiation.cc:299] Negotiation successful
0806 19:38:37.534549 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 19:38:37.542711 23715 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48094:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:38:37.557438 23715 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:38:37.652011 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:37.610431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:36431 (local address 127.0.0.1:53544)
0806 19:38:37.624326 (+ 13895us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:37.624378 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:37.624625 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:37.635300 (+ 10675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:37.635310 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:37.635356 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:37.635882 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:37.635902 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.642564 (+ 6662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.642572 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:37.645251 (+ 2679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:37.645282 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.650871 (+ 5589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.650882 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:37.651143 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:37.651221 (+ 78us) client_negotiation.cc:769] Sending connection context
0806 19:38:37.651361 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 19:38:37.651613 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13546,"threads_started":1}
I20260806 19:38:37.667088 24224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:37.623275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:42039 (local address 127.0.0.1:49136)
0806 19:38:37.652747 (+ 29472us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:37.652773 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:37.652934 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:37.657737 (+ 4803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:37.657744 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:37.657778 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:37.658161 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:37.658176 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.660411 (+ 2235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.660416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:37.662568 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:37.662586 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.666246 (+ 3660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.666253 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:37.666449 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:37.666487 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 19:38:37.666587 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 19:38:37.666740 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29315,"thread_start_us":1022,"threads_started":1}
I20260806 19:38:37.669708 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:37.624089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49136 (local address 127.10.25.1:42039)
0806 19:38:37.655089 (+ 31000us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:37.655099 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:37.655132 (+ 33us) server_negotiation.cc:407] Connection header received
0806 19:38:37.655240 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:37.655249 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:37.655540 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:37.655760 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:37.658409 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.660211 (+ 1802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.662789 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.664212 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.664342 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:37.664568 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:37.669078 (+ 4510us) server_negotiation.cc:299] Negotiation successful
0806 19:38:37.669305 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29321,"thread_start_us":29398,"threads_started":1}
I20260806 19:38:37.673895 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:37.610725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53544 (local address 127.10.25.3:36431)
0806 19:38:37.633805 (+ 23080us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:37.633816 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:37.633869 (+ 53us) server_negotiation.cc:407] Connection header received
0806 19:38:37.634185 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:37.634196 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:37.634594 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:37.637793 (+ 3199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:37.638184 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.642290 (+ 4106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.647071 (+ 4781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.650427 (+ 3356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.672909 (+ 22482us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:37.673204 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:37.673305 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 19:38:37.673531 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19535,"thread_start_us":19348,"threads_started":1}
I20260806 19:38:37.676474 23852 tablet_service.cc:1467] Processing CreateTablet for tablet d615d751ac4c4f51acb0130c6d9ece3d (DEFAULT_TABLE table=test-table [id=a8f8b1056bc04db8858fdcc5dcaf2209]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:38:37.677831 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:37.614217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:46197 (local address 127.0.0.1:60240)
0806 19:38:37.652326 (+ 38109us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:37.652348 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:37.652504 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:37.654038 (+ 1534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:37.654043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:37.654073 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:37.654266 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:37.654278 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.658749 (+ 4471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.658755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:37.661321 (+ 2566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:37.661344 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.676926 (+ 15582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.676939 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:37.677169 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:37.677206 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 19:38:37.677306 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 19:38:37.677462 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37958,"thread_start_us":39507,"threads_started":1}
I20260806 19:38:37.683130 23852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d615d751ac4c4f51acb0130c6d9ece3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:38:37.680943 24135 tablet_service.cc:1467] Processing CreateTablet for tablet d615d751ac4c4f51acb0130c6d9ece3d (DEFAULT_TABLE table=test-table [id=a8f8b1056bc04db8858fdcc5dcaf2209]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:38:37.684731 24135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d615d751ac4c4f51acb0130c6d9ece3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:38:37.687732 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:37.614376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60240 (local address 127.10.25.2:46197)
0806 19:38:37.629832 (+ 15456us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:37.629846 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:37.652804 (+ 22958us) server_negotiation.cc:407] Connection header received
0806 19:38:37.652951 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:37.652961 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:37.653272 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:37.653551 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:37.654478 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.656164 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.661646 (+ 5482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:37.663467 (+ 1821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:37.663658 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:37.663952 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:37.685032 (+ 21080us) server_negotiation.cc:299] Negotiation successful
0806 19:38:37.685281 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15244,"thread_start_us":15048,"threads_started":1}
I20260806 19:38:37.694093 23998 tablet_service.cc:1467] Processing CreateTablet for tablet d615d751ac4c4f51acb0130c6d9ece3d (DEFAULT_TABLE table=test-table [id=a8f8b1056bc04db8858fdcc5dcaf2209]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:38:37.696410 23998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d615d751ac4c4f51acb0130c6d9ece3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:38:37.698022 24065 heartbeater.cc:503] Master 127.10.25.62:39073 was elected leader, sending a full tablet report...
I20260806 19:38:37.722491 24228 tablet_bootstrap.cc:492] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578: Bootstrap starting.
I20260806 19:38:37.728775 24228 tablet_bootstrap.cc:654] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578: Neither blocks nor log segments found. Creating new log.
I20260806 19:38:37.731340 24229 tablet_bootstrap.cc:492] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6: Bootstrap starting.
I20260806 19:38:37.737949 24229 tablet_bootstrap.cc:654] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6: Neither blocks nor log segments found. Creating new log.
I20260806 19:38:37.739728 24228 log.cc:824] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:37.741299 24229 log.cc:824] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:37.751586 24229 tablet_bootstrap.cc:492] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6: No bootstrap required, opened a new log
I20260806 19:38:37.754841 24228 tablet_bootstrap.cc:492] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578: No bootstrap required, opened a new log
I20260806 19:38:37.755290 24228 ts_tablet_manager.cc:1397] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578: Time spent bootstrapping tablet: real 0.033s user 0.008s sys 0.005s
I20260806 19:38:37.757903 24229 ts_tablet_manager.cc:1397] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6: Time spent bootstrapping tablet: real 0.027s user 0.008s sys 0.004s
I20260806 19:38:37.770437 24230 tablet_bootstrap.cc:492] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67: Bootstrap starting.
I20260806 19:38:37.777451 24229 raft_consensus.cc:348] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46b035327cbd491285cdd69526ed2578" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36431 } } peers { permanent_uuid: "082c1c1cab2d478ca367f57f044febf6" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46197 } } peers { permanent_uuid: "ac46213f362c4733b8a698f020038d67" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42039 } }
I20260806 19:38:37.778352 24229 raft_consensus.cc:374] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:38:37.778553 24229 raft_consensus.cc:729] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 082c1c1cab2d478ca367f57f044febf6, State: Initialized, Role: FOLLOWER
I20260806 19:38:37.779647 24229 consensus_queue.cc:260] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46b035327cbd491285cdd69526ed2578" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36431 } } peers { permanent_uuid: "082c1c1cab2d478ca367f57f044febf6" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46197 } } peers { permanent_uuid: "ac46213f362c4733b8a698f020038d67" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42039 } }
I20260806 19:38:37.798238 24230 tablet_bootstrap.cc:654] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67: Neither blocks nor log segments found. Creating new log.
I20260806 19:38:37.799983 24230 log.cc:824] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:37.801613 24228 raft_consensus.cc:348] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46b035327cbd491285cdd69526ed2578" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36431 } } peers { permanent_uuid: "082c1c1cab2d478ca367f57f044febf6" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46197 } } peers { permanent_uuid: "ac46213f362c4733b8a698f020038d67" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42039 } }
I20260806 19:38:37.802317 24228 raft_consensus.cc:374] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:38:37.802474 24228 raft_consensus.cc:729] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46b035327cbd491285cdd69526ed2578, State: Initialized, Role: FOLLOWER
I20260806 19:38:37.803579 24228 consensus_queue.cc:260] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46b035327cbd491285cdd69526ed2578" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36431 } } peers { permanent_uuid: "082c1c1cab2d478ca367f57f044febf6" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46197 } } peers { permanent_uuid: "ac46213f362c4733b8a698f020038d67" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42039 } }
I20260806 19:38:37.806892 24229 ts_tablet_manager.cc:1428] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6: Time spent starting tablet: real 0.049s user 0.026s sys 0.011s
I20260806 19:38:37.810075 24228 ts_tablet_manager.cc:1428] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578: Time spent starting tablet: real 0.055s user 0.016s sys 0.002s
I20260806 19:38:37.814819 24230 tablet_bootstrap.cc:492] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67: No bootstrap required, opened a new log
I20260806 19:38:37.815224 24230 ts_tablet_manager.cc:1397] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67: Time spent bootstrapping tablet: real 0.045s user 0.011s sys 0.000s
I20260806 19:38:37.813886 24205 heartbeater.cc:503] Master 127.10.25.62:39073 was elected leader, sending a full tablet report...
I20260806 19:38:37.828552 24230 raft_consensus.cc:348] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46b035327cbd491285cdd69526ed2578" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36431 } } peers { permanent_uuid: "082c1c1cab2d478ca367f57f044febf6" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46197 } } peers { permanent_uuid: "ac46213f362c4733b8a698f020038d67" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42039 } }
I20260806 19:38:37.829164 24230 raft_consensus.cc:374] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:38:37.829288 24230 raft_consensus.cc:729] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac46213f362c4733b8a698f020038d67, State: Initialized, Role: FOLLOWER
I20260806 19:38:37.830163 24230 consensus_queue.cc:260] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46b035327cbd491285cdd69526ed2578" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36431 } } peers { permanent_uuid: "082c1c1cab2d478ca367f57f044febf6" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46197 } } peers { permanent_uuid: "ac46213f362c4733b8a698f020038d67" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42039 } }
I20260806 19:38:37.831729 23918 heartbeater.cc:503] Master 127.10.25.62:39073 was elected leader, sending a full tablet report...
I20260806 19:38:37.832610 24230 ts_tablet_manager.cc:1428] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67: Time spent starting tablet: real 0.017s user 0.011s sys 0.006s
W20260806 19:38:37.860459 24066 tablet.cc:2369] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:38:37.943208 24206 tablet.cc:2369] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:38:38.035128 24233 raft_consensus.cc:482] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:38:38.035940 24233 raft_consensus.cc:504] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46b035327cbd491285cdd69526ed2578" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36431 } } peers { permanent_uuid: "082c1c1cab2d478ca367f57f044febf6" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46197 } } peers { permanent_uuid: "ac46213f362c4733b8a698f020038d67" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42039 } }
I20260806 19:38:38.038496 24235 raft_consensus.cc:482] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:38:38.038954 24233 leader_election.cc:288] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46b035327cbd491285cdd69526ed2578 (127.10.25.3:36431), ac46213f362c4733b8a698f020038d67 (127.10.25.1:42039)
I20260806 19:38:38.038859 24235 raft_consensus.cc:504] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46b035327cbd491285cdd69526ed2578" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36431 } } peers { permanent_uuid: "082c1c1cab2d478ca367f57f044febf6" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46197 } } peers { permanent_uuid: "ac46213f362c4733b8a698f020038d67" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42039 } }
W20260806 19:38:38.041280 23919 tablet.cc:2369] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:38:38.042501 24235 leader_election.cc:288] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 082c1c1cab2d478ca367f57f044febf6 (127.10.25.2:46197), ac46213f362c4733b8a698f020038d67 (127.10.25.1:42039)
I20260806 19:38:38.058642 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.039945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:33589 (local address 127.10.25.1:42039)
0806 19:38:38.040173 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:38.040180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:38.040940 (+ 760us) server_negotiation.cc:407] Connection header received
0806 19:38:38.041328 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:38.041337 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:38.041458 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:38.041574 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:38.043107 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.044478 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.055414 (+ 10936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.056914 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.057091 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:38.057292 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:38.057935 (+ 643us) server_negotiation.cc:299] Negotiation successful
0806 19:38:38.058079 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":51}
I20260806 19:38:38.059904 24245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.039300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:42039 (local address 127.10.25.2:33589)
0806 19:38:38.040823 (+ 1523us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:38.040845 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:38.041105 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:38.041828 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:38.041833 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:38.041875 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:38.042862 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:38.042882 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.050907 (+ 8025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.050918 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:38.054419 (+ 3501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:38.054443 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.057523 (+ 3080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.057531 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:38.057721 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:38.057758 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 19:38:38.059276 (+ 1518us) client_negotiation.cc:240] Negotiation successful
0806 19:38:38.059469 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1366,"thread_start_us":14823,"threads_started":1}
I20260806 19:38:38.061616 23872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d615d751ac4c4f51acb0130c6d9ece3d" candidate_uuid: "082c1c1cab2d478ca367f57f044febf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac46213f362c4733b8a698f020038d67" is_pre_election: true
I20260806 19:38:38.062407 23872 raft_consensus.cc:2454] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 082c1c1cab2d478ca367f57f044febf6 in term 0.
I20260806 19:38:38.063002 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.054482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:50929 (local address 127.10.25.2:46197)
0806 19:38:38.054667 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:38.054683 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:38.054702 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:38:38.054773 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:38.054779 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:38.054895 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:38.054990 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:38.055868 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.057197 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.060212 (+ 3015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.061698 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.061818 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:38.061948 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:38.062562 (+ 614us) server_negotiation.cc:299] Negotiation successful
0806 19:38:38.062732 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 19:38:38.063211 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.039847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:37433 (local address 127.10.25.3:36431)
0806 19:38:38.040140 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:38.040149 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:38.054613 (+ 14464us) server_negotiation.cc:407] Connection header received
0806 19:38:38.054990 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:38.054996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:38.055072 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:38.055141 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:38.056216 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.057549 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.060268 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.061887 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.062025 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:38.062128 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:38.062742 (+ 614us) server_negotiation.cc:299] Negotiation successful
0806 19:38:38.062893 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":77,"spinlock_wait_cycles":28544}
I20260806 19:38:38.064147 24244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.039373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:36431 (local address 127.10.25.2:37433)
0806 19:38:38.054500 (+ 15127us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:38.054521 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:38.054766 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:38.055454 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:38.055460 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:38.055491 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:38.055986 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:38.056007 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.057762 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.057769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:38.060073 (+ 2304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:38.060096 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.062342 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.062349 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:38.062558 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:38.062594 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 19:38:38.063471 (+ 877us) client_negotiation.cc:240] Negotiation successful
0806 19:38:38.063638 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14974,"mutex_wait_us":1,"thread_start_us":2766,"threads_started":1}
I20260806 19:38:38.064073 23950 leader_election.cc:302] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 082c1c1cab2d478ca367f57f044febf6, ac46213f362c4733b8a698f020038d67; no voters:
I20260806 19:38:38.064798 24233 raft_consensus.cc:2781] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:38:38.065002 24233 raft_consensus.cc:482] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:38:38.065121 24233 raft_consensus.cc:3037] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:38:38.065289 24247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.043161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:42039 (local address 127.10.25.3:52833)
0806 19:38:38.054478 (+ 11317us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:38.054501 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:38.054635 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:38.055922 (+ 1287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:38.055928 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:38.055956 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:38.056419 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:38.056434 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.058732 (+ 2298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.058738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:38.061415 (+ 2677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:38.061435 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.064064 (+ 2629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.064078 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:38.064334 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:38.064382 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 19:38:38.064519 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 19:38:38.064704 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11174,"mutex_wait_us":38,"thread_start_us":11588,"threads_started":1}
I20260806 19:38:38.066491 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.044795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:52833 (local address 127.10.25.1:42039)
0806 19:38:38.055573 (+ 10778us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:38.055578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:38.055604 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:38:38.055677 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:38.055682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:38.055753 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:38.056593 (+ 840us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:38.056948 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.058502 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.061729 (+ 3227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.063574 (+ 1845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.064065 (+ 491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:38.064247 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:38.064589 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 19:38:38.064806 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":10615,"thread_start_us":10343,"threads_started":1}
I20260806 19:38:38.066591 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.041631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:46197 (local address 127.10.25.3:50929)
0806 19:38:38.053166 (+ 11535us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:38.053197 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:38.053419 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:38.055181 (+ 1762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:38.055186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:38.055226 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:38.055586 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:38.055602 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.057405 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.057411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:38.059969 (+ 2558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:38.059995 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.062093 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.062099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:38.062338 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:38.062373 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 19:38:38.063530 (+ 1157us) client_negotiation.cc:240] Negotiation successful
0806 19:38:38.063683 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11278,"mutex_wait_us":1,"thread_start_us":11928,"threads_started":1}
I20260806 19:38:38.065948 24155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d615d751ac4c4f51acb0130c6d9ece3d" candidate_uuid: "082c1c1cab2d478ca367f57f044febf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46b035327cbd491285cdd69526ed2578" is_pre_election: true
I20260806 19:38:38.067013 24155 raft_consensus.cc:2454] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 082c1c1cab2d478ca367f57f044febf6 in term 0.
I20260806 19:38:38.067265 23872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d615d751ac4c4f51acb0130c6d9ece3d" candidate_uuid: "46b035327cbd491285cdd69526ed2578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac46213f362c4733b8a698f020038d67" is_pre_election: true
I20260806 19:38:38.067569 23872 raft_consensus.cc:2454] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46b035327cbd491285cdd69526ed2578 in term 0.
I20260806 19:38:38.067757 24018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d615d751ac4c4f51acb0130c6d9ece3d" candidate_uuid: "46b035327cbd491285cdd69526ed2578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "082c1c1cab2d478ca367f57f044febf6" is_pre_election: true
I20260806 19:38:38.068161 24086 leader_election.cc:302] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46b035327cbd491285cdd69526ed2578, ac46213f362c4733b8a698f020038d67; no voters:
I20260806 19:38:38.068883 24235 raft_consensus.cc:2781] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:38:38.069100 24235 raft_consensus.cc:482] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:38:38.069211 24235 raft_consensus.cc:3037] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:38:38.072063 24233 raft_consensus.cc:504] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46b035327cbd491285cdd69526ed2578" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36431 } } peers { permanent_uuid: "082c1c1cab2d478ca367f57f044febf6" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46197 } } peers { permanent_uuid: "ac46213f362c4733b8a698f020038d67" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42039 } }
I20260806 19:38:38.072733 24018 raft_consensus.cc:2379] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 46b035327cbd491285cdd69526ed2578 in current term 1: Already voted for candidate 082c1c1cab2d478ca367f57f044febf6 in this term.
I20260806 19:38:38.072911 24235 raft_consensus.cc:504] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46b035327cbd491285cdd69526ed2578" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36431 } } peers { permanent_uuid: "082c1c1cab2d478ca367f57f044febf6" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46197 } } peers { permanent_uuid: "ac46213f362c4733b8a698f020038d67" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42039 } }
I20260806 19:38:38.073174 24233 leader_election.cc:288] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [CANDIDATE]: Term 1 election: Requested vote from peers 46b035327cbd491285cdd69526ed2578 (127.10.25.3:36431), ac46213f362c4733b8a698f020038d67 (127.10.25.1:42039)
I20260806 19:38:38.073962 24155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d615d751ac4c4f51acb0130c6d9ece3d" candidate_uuid: "082c1c1cab2d478ca367f57f044febf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46b035327cbd491285cdd69526ed2578"
I20260806 19:38:38.074146 23872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d615d751ac4c4f51acb0130c6d9ece3d" candidate_uuid: "082c1c1cab2d478ca367f57f044febf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac46213f362c4733b8a698f020038d67"
I20260806 19:38:38.074268 24155 raft_consensus.cc:2379] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 082c1c1cab2d478ca367f57f044febf6 in current term 1: Already voted for candidate 46b035327cbd491285cdd69526ed2578 in this term.
I20260806 19:38:38.074424 23872 raft_consensus.cc:3037] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:38:38.074615 23871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d615d751ac4c4f51acb0130c6d9ece3d" candidate_uuid: "46b035327cbd491285cdd69526ed2578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac46213f362c4733b8a698f020038d67"
I20260806 19:38:38.075654 24018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d615d751ac4c4f51acb0130c6d9ece3d" candidate_uuid: "46b035327cbd491285cdd69526ed2578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "082c1c1cab2d478ca367f57f044febf6"
I20260806 19:38:38.075855 24018 raft_consensus.cc:2379] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 46b035327cbd491285cdd69526ed2578 in current term 1: Already voted for candidate 082c1c1cab2d478ca367f57f044febf6 in this term.
I20260806 19:38:38.075855 24235 leader_election.cc:288] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [CANDIDATE]: Term 1 election: Requested vote from peers 082c1c1cab2d478ca367f57f044febf6 (127.10.25.2:46197), ac46213f362c4733b8a698f020038d67 (127.10.25.1:42039)
I20260806 19:38:38.076217 24088 leader_election.cc:302] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 46b035327cbd491285cdd69526ed2578; no voters: 082c1c1cab2d478ca367f57f044febf6, ac46213f362c4733b8a698f020038d67
I20260806 19:38:38.076684 24235 raft_consensus.cc:2726] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 19:38:38.078099 23872 raft_consensus.cc:2454] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 082c1c1cab2d478ca367f57f044febf6 in term 1.
I20260806 19:38:38.078699 23950 leader_election.cc:302] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 082c1c1cab2d478ca367f57f044febf6, ac46213f362c4733b8a698f020038d67; no voters: 46b035327cbd491285cdd69526ed2578
I20260806 19:38:38.079113 24233 raft_consensus.cc:2781] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:38:38.079644 24233 raft_consensus.cc:686] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [term 1 LEADER]: Becoming Leader. State: Replica: 082c1c1cab2d478ca367f57f044febf6, State: Running, Role: LEADER
I20260806 19:38:38.080282 24233 consensus_queue.cc:237] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46b035327cbd491285cdd69526ed2578" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36431 } } peers { permanent_uuid: "082c1c1cab2d478ca367f57f044febf6" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46197 } } peers { permanent_uuid: "ac46213f362c4733b8a698f020038d67" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42039 } }
I20260806 19:38:38.084995 23714 catalog_manager.cc:5495] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 082c1c1cab2d478ca367f57f044febf6 (127.10.25.2). New cstate: current_term: 1 leader_uuid: "082c1c1cab2d478ca367f57f044febf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46b035327cbd491285cdd69526ed2578" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "082c1c1cab2d478ca367f57f044febf6" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 46197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac46213f362c4733b8a698f020038d67" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42039 } health_report { overall_health: UNKNOWN } } }
I20260806 19:38:38.149029 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.141695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60256 (local address 127.10.25.2:46197)
0806 19:38:38.141891 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:38.141896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:38.142432 (+ 536us) server_negotiation.cc:407] Connection header received
0806 19:38:38.142632 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:38.142636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:38.142757 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:38.142909 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:38:38.143386 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.144161 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.146373 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.146988 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.147148 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:38.147737 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:38:38.148320 (+ 583us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:38:38.148529 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:38.148585 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 19:38:38.148693 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 19:38:38.158806 23872 raft_consensus.cc:1261] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67 [term 1 FOLLOWER]: Refusing update from remote peer 082c1c1cab2d478ca367f57f044febf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:38:38.158877 24155 raft_consensus.cc:1261] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [term 1 FOLLOWER]: Refusing update from remote peer 082c1c1cab2d478ca367f57f044febf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:38:38.159933 24249 consensus_queue.cc:1035] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46b035327cbd491285cdd69526ed2578" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:38:38.160338 24233 consensus_queue.cc:1035] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac46213f362c4733b8a698f020038d67" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 42039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:38:38.196620 23715 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:38:38.197005 23715 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:38:38.202196 23715 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3896fb0a982d4ac88cd529debb4e97b1: Sending DeleteTablet for 3 replicas of tablet d615d751ac4c4f51acb0130c6d9ece3d
I20260806 19:38:38.203758 23998 tablet_service.cc:1514] Processing DeleteTablet for tablet d615d751ac4c4f51acb0130c6d9ece3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:38:38 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:60240
I20260806 19:38:38.204116 23852 tablet_service.cc:1514] Processing DeleteTablet for tablet d615d751ac4c4f51acb0130c6d9ece3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:38:38 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:49136
I20260806 19:38:38.204682 24135 tablet_service.cc:1514] Processing DeleteTablet for tablet d615d751ac4c4f51acb0130c6d9ece3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:38:38 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:53544
I20260806 19:38:38.208777 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 23754
I20260806 19:38:38.213356 24264 tablet_replica.cc:330] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6: stopping tablet replica
I20260806 19:38:38.214052 24264 raft_consensus.cc:2229] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:38:38.215103 24264 raft_consensus.cc:2258] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:38:38.217926 24264 ts_tablet_manager.cc:1905] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:38:38.217996 24265 tablet_replica.cc:330] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67: stopping tablet replica
I20260806 19:38:38.218564 24265 raft_consensus.cc:2229] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:38:38.219184 24265 raft_consensus.cc:2258] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:38:38.220353 24266 tablet_replica.cc:330] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578: stopping tablet replica
I20260806 19:38:38.221010 24266 raft_consensus.cc:2229] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:38:38.221733 24266 raft_consensus.cc:2258] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:38:38.224404 24266 ts_tablet_manager.cc:1905] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:38:38.227766 24265 ts_tablet_manager.cc:1905] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:38:38.230417 24264 ts_tablet_manager.cc:1918] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:38:38.232499 24266 ts_tablet_manager.cc:1918] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:38:38.232810 24266 log.cc:1196] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/d615d751ac4c4f51acb0130c6d9ece3d
I20260806 19:38:38.233150 24264 log.cc:1196] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/d615d751ac4c4f51acb0130c6d9ece3d
I20260806 19:38:38.233583 24266 ts_tablet_manager.cc:1939] T d615d751ac4c4f51acb0130c6d9ece3d P 46b035327cbd491285cdd69526ed2578: Deleting consensus metadata
I20260806 19:38:38.234242 24264 ts_tablet_manager.cc:1939] T d615d751ac4c4f51acb0130c6d9ece3d P 082c1c1cab2d478ca367f57f044febf6: Deleting consensus metadata
I20260806 19:38:38.236063 23702 catalog_manager.cc:4841] TS 46b035327cbd491285cdd69526ed2578 (127.10.25.3:36431): tablet d615d751ac4c4f51acb0130c6d9ece3d (table test-table [id=a8f8b1056bc04db8858fdcc5dcaf2209]) successfully deleted
I20260806 19:38:38.240541 23701 catalog_manager.cc:4841] TS 082c1c1cab2d478ca367f57f044febf6 (127.10.25.2:46197): tablet d615d751ac4c4f51acb0130c6d9ece3d (table test-table [id=a8f8b1056bc04db8858fdcc5dcaf2209]) successfully deleted
I20260806 19:38:38.242105 24265 ts_tablet_manager.cc:1918] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:38:38.242336 24265 log.cc:1196] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/d615d751ac4c4f51acb0130c6d9ece3d
I20260806 19:38:38.243011 24265 ts_tablet_manager.cc:1939] T d615d751ac4c4f51acb0130c6d9ece3d P ac46213f362c4733b8a698f020038d67: Deleting consensus metadata
I20260806 19:38:38.243363 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.212807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49152 (local address 127.10.25.1:42039)
0806 19:38:38.213065 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:38.213072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:38.222145 (+ 9073us) server_negotiation.cc:407] Connection header received
0806 19:38:38.222493 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:38.222500 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:38.222594 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:38.222677 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:38.225081 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.226376 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.228549 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.229305 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.229425 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:38.233877 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:38.233900 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:38.233923 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:38.233984 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:38.238698 (+ 4714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:38.239828 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:38.239841 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:38.239847 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:38.240058 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:38.240649 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:38.240654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:38.240657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:38.241135 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:38.241471 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:38.242490 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0806 19:38:38.242805 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 19:38:38.244124 23912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:38.350282 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 23754
W20260806 19:38:38.371840 23699 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.25.1:42039 (error 108)
I20260806 19:38:38.372836 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 23922
I20260806 19:38:38.374650 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.373456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:42039 (local address 127.0.0.1:49168)
0806 19:38:38.374207 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:38.374359 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:42039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":442,"threads_started":1}
W20260806 19:38:38.375023 23699 catalog_manager.cc:4584] TS ac46213f362c4733b8a698f020038d67 (127.10.25.1:42039): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d615d751ac4c4f51acb0130c6d9ece3d: Network error: Client connection negotiation failed: client connection to 127.10.25.1:42039: connect: Connection refused (error 111)
I20260806 19:38:38.393712 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.373780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60270 (local address 127.10.25.2:46197)
0806 19:38:38.373996 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:38.374003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:38.374329 (+ 326us) server_negotiation.cc:407] Connection header received
0806 19:38:38.374535 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:38.374541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:38.374643 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:38.374773 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:38.376887 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.377996 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.380210 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.380830 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.380992 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:38.384837 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:38.384851 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:38.384867 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:38.384914 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:38.388951 (+ 4037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:38.390131 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:38.390140 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:38.390143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:38.390340 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:38.390824 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:38.390828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:38.390830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:38.391320 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:38.392093 (+ 773us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:38.392668 (+ 575us) server_negotiation.cc:299] Negotiation successful
0806 19:38:38.393277 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 19:38:38.394606 24059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:38.429245 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.428566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:42039 (local address 127.0.0.1:49182)
0806 19:38:38.428882 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:38.429052 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:42039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260806 19:38:38.430905 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.430368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:42039 (local address 127.0.0.1:49188)
0806 19:38:38.430619 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:38.430708 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:42039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260806 19:38:38.491986 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.491396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:42039 (local address 127.0.0.1:49192)
0806 19:38:38.491669 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:38.491784 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:42039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260806 19:38:38.493530 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.492950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:42039 (local address 127.0.0.1:49208)
0806 19:38:38.493162 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:38.493255 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:42039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 19:38:38.530468 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 23922
I20260806 19:38:38.553736 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 24069
I20260806 19:38:38.572403 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.554662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53554 (local address 127.10.25.3:36431)
0806 19:38:38.554888 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:38.554894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:38.554975 (+ 81us) server_negotiation.cc:407] Connection header received
0806 19:38:38.555133 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:38.555137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:38.555204 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:38.555296 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:38.556859 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.557764 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.559385 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:38.559842 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:38.559953 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:38.564124 (+ 4171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:38.564143 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:38.564160 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:38.564210 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:38.568420 (+ 4210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:38.569800 (+ 1380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:38.569810 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:38.569813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:38.570003 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:38.570607 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:38.570611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:38.570614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:38.571040 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:38.571299 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:38.571844 (+ 545us) server_negotiation.cc:299] Negotiation successful
0806 19:38:38.572065 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 19:38:38.573132 24198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:38.574424 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.573865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:42039 (local address 127.0.0.1:49210)
0806 19:38:38.574072 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:38.574241 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:42039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 19:38:38.576057 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.575526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:42039 (local address 127.0.0.1:49216)
0806 19:38:38.575746 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:38.575842 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:42039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 19:38:38.671736 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 24069
I20260806 19:38:38.692510 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 23682
I20260806 19:38:38.694192 23747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:38.823808 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:38.823370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:42039 (local address 127.0.0.1:49226)
0806 19:38:38.823557 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:38.823655 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:42039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 19:38:38.824685 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 23682
I20260806 19:38:38.857774 10340 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 19:38:39.205725 10340 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 19:38:39.206049 10340 mini_ranger.cc:66] Stopping Ranger...
I20260806 19:38:39.553992 10340 mini_ranger.cc:68] Stopped Ranger
2026-08-06 19:38:39.554 UTC [22725] LOG: received smart shutdown request
2026-08-06 19:38:39.559 UTC [22725] LOG: background worker "logical replication launcher" (PID 22734) exited with exit code 1
2026-08-06 19:38:39.559 UTC [22729] LOG: shutting down
2026-08-06 19:38:39.812 UTC [22725] LOG: database system is shut down
2026-08-06T19:38:39Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (67174 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:38:40 dist-test-slave-ss4n krb5kdc[24283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:38:40 dist-test-slave-ss4n krb5kdc[24283](info): set up 2 sockets
Aug 06 19:38:40 dist-test-slave-ss4n krb5kdc[24283](info): commencing operation
krb5kdc: starting...
W20260806 19:38:41.196498 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:38:41 dist-test-slave-ss4n krb5kdc[24283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:38:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:38:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:38:41.519030 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:37499
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:37499,127.10.25.61:36915,127.10.25.60:34529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:38:41.920076 24296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:41.920881 24296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:41.921113 24296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:41.933188 24296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:38:41.933364 24296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:41.933532 24296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:41.933669 24296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:38:41.933768 24296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:38:41.947002 24296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:37499,127.10.25.61:36915,127.10.25.60:34529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:37499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:41.948637 24296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:41.950266 24296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:41.964541 24304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:41.964462 24302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:41.964481 24301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:41.965417 24296 server_base.cc:1034] running on GCE node
I20260806 19:38:41.966475 24296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:41.968354 24296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:41.969646 24296 hybrid_clock.cc:648] HybridClock initialized: now 1786045121969592 us; error 58 us; skew 500 ppm
I20260806 19:38:41.970122 24296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:41 dist-test-slave-ss4n krb5kdc[24283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045121, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:41.974172 24296 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:38:41.976383 24296 webserver.cc:460] Webserver started at http://127.10.25.62:34227/ using document root <none> and password file <none>
I20260806 19:38:41.977222 24296 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:41.977396 24296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:41.977854 24296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:38:41.981266 24296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "1cba0c6d82d243d18914c2e323698041"
format_stamp: "Formatted at 2026-08-06 19:38:41 on dist-test-slave-ss4n"
server_key: "2bdc34f0540ae89d5c471b3bf018a306"
server_key_iv: "28735c9df111f7b6022b590e8226b95a"
server_key_version: "encryptionkey@0"
I20260806 19:38:41.982323 24296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "1cba0c6d82d243d18914c2e323698041"
format_stamp: "Formatted at 2026-08-06 19:38:41 on dist-test-slave-ss4n"
server_key: "2bdc34f0540ae89d5c471b3bf018a306"
server_key_iv: "28735c9df111f7b6022b590e8226b95a"
server_key_version: "encryptionkey@0"
I20260806 19:38:41.988492 24296 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:38:41.993175 24311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:41.994935 24296 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:38:41.995203 24296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "1cba0c6d82d243d18914c2e323698041"
format_stamp: "Formatted at 2026-08-06 19:38:41 on dist-test-slave-ss4n"
server_key: "2bdc34f0540ae89d5c471b3bf018a306"
server_key_iv: "28735c9df111f7b6022b590e8226b95a"
server_key_version: "encryptionkey@0"
I20260806 19:38:41.995641 24296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:42.018626 24296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:42.024116 24296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:42.024477 24296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:42.048488 24296 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:37499
I20260806 19:38:42.048537 24363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:37499 every 8 connection(s)
I20260806 19:38:42.050365 24296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:38:42.054792 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 24296
I20260806 19:38:42.055017 24364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:38:42.055045 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:38:42.056209 10340 external_mini_cluster.cc:1428] Setting key 01f61eda7e20c2b7766d3111da32892c
I20260806 19:38:42.068317 24364 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:38:42 dist-test-slave-ss4n krb5kdc[24283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045121, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.25.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.61@KRBTEST.COM" created.
I20260806 19:38:42.096213 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.074380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36378 (local address 127.10.25.62:37499)
0806 19:38:42.075331 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:42.075341 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:42.075428 (+ 87us) server_negotiation.cc:407] Connection header received
0806 19:38:42.076851 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:42.076858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:42.077154 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:42.077406 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:42.078995 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:42.080398 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:42.082161 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:42.082869 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:42.083117 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:42.087542 (+ 4425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:42.087552 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:42.087567 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:42.087604 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:42.091273 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:42.092229 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:42.092237 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:42.092240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:42.092413 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:42.092857 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:42.092864 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:42.092872 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:42.093437 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:42.093712 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:42.094259 (+ 547us) server_negotiation.cc:299] Negotiation successful
0806 19:38:42.094652 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":701,"thread_start_us":486,"threads_started":1}
I20260806 19:38:42.096251 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.072594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:37499 (local address 127.0.0.1:36378)
0806 19:38:42.073748 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.073807 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:42.076346 (+ 2539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:42.077436 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:42.077445 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:42.078349 (+ 904us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:42.078738 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:42.078758 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:42.080600 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:42.080606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:42.081985 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:42.082001 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:42.083187 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:42.083195 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:42.083480 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:42.084389 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:42.084415 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:42.087227 (+ 2812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:42.091448 (+ 4221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:42.091462 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:42.091476 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:42.092068 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:42.092535 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:42.092542 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:42.092546 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:42.092723 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:42.093630 (+ 907us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:42.093638 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:42.093873 (+ 235us) client_negotiation.cc:769] Sending connection context
0806 19:38:42.094129 (+ 256us) client_negotiation.cc:240] Negotiation successful
0806 19:38:42.094784 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":226,"thread_start_us":492,"threads_started":1}
I20260806 19:38:42.100590 24364 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:38:42.102116 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.101619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:36915 (local address 127.0.0.1:50434)
0806 19:38:42.101808 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.101976 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 19:38:42.102452 24315 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.25.61:36915: connect: Connection refused (error 111)
I20260806 19:38:42.104528 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.104069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:36915 (local address 127.0.0.1:50450)
0806 19:38:42.104269 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.104358 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 19:38:42.104887 24364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.61:36915: Network error: Client connection negotiation failed: client connection to 127.10.25.61:36915: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.61.keytab.
Entry for principal kudu/127.10.25.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.61.keytab.
I20260806 19:38:42.130586 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.61:36915
--webserver_interface=127.10.25.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.61.keytab
--principal=kudu/127.10.25.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:37499,127.10.25.61:36915,127.10.25.60:34529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:38:42.154501 24364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } attempt: 1
I20260806 19:38:42.155771 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.155292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:36915 (local address 127.0.0.1:50464)
0806 19:38:42.155506 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.155639 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 19:38:42.157213 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.156657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:36915 (local address 127.0.0.1:50468)
0806 19:38:42.156862 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.156951 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260806 19:38:42.157608 24364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.61:36915: Network error: Client connection negotiation failed: client connection to 127.10.25.61:36915: connect: Connection refused (error 111)
I20260806 19:38:42.225981 24364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } attempt: 2
I20260806 19:38:42.227296 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.226791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:36915 (local address 127.0.0.1:50482)
0806 19:38:42.227023 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.227139 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260806 19:38:42.228513 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.228057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:36915 (local address 127.0.0.1:50486)
0806 19:38:42.228260 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.228356 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260806 19:38:42.228739 24364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.61:36915: Network error: Client connection negotiation failed: client connection to 127.10.25.61:36915: connect: Connection refused (error 111)
I20260806 19:38:42.320058 24364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } attempt: 3
I20260806 19:38:42.321424 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.320913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:36915 (local address 127.0.0.1:50496)
0806 19:38:42.321173 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.321286 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260806 19:38:42.322901 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.322411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:36915 (local address 127.0.0.1:50508)
0806 19:38:42.322631 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.322713 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260806 19:38:42.323173 24364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.61:36915: Network error: Client connection negotiation failed: client connection to 127.10.25.61:36915: connect: Connection refused (error 111)
I20260806 19:38:42.466481 24364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } attempt: 4
I20260806 19:38:42.468120 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.467366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:36915 (local address 127.0.0.1:50512)
0806 19:38:42.467860 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.467956 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327}
I20260806 19:38:42.469419 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.468950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:36915 (local address 127.0.0.1:50520)
0806 19:38:42.469165 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.469279 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:36915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260806 19:38:42.469834 24364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.61:36915: Network error: Client connection negotiation failed: client connection to 127.10.25.61:36915: connect: Connection refused (error 111)
W20260806 19:38:42.516451 24371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:42.516865 24371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:42.517032 24371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:42.524942 24371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:38:42.525139 24371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:42.525256 24371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:42.525365 24371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:38:42.525447 24371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:38:42.536283 24371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:37499,127.10.25.61:36915,127.10.25.60:34529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.61.keytab
--principal=kudu/127.10.25.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.61:36915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.25.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:42.537957 24371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:42.539455 24371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:42.555891 24377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:42.556499 24379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:42.556644 24376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:42.557156 24371 server_base.cc:1034] running on GCE node
I20260806 19:38:42.558161 24371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:42.560037 24371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:42.561657 24371 hybrid_clock.cc:648] HybridClock initialized: now 1786045122561368 us; error 81 us; skew 500 ppm
I20260806 19:38:42.562404 24371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:42 dist-test-slave-ss4n krb5kdc[24283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045122, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:42.566666 24371 init.cc:377] Logged in from keytab as kudu/127.10.25.61@KRBTEST.COM (short username kudu)
I20260806 19:38:42.568706 24371 webserver.cc:460] Webserver started at http://127.10.25.61:39011/ using document root <none> and password file <none>
I20260806 19:38:42.569705 24371 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:42.569847 24371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:42.570266 24371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:38:42.574129 24371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/data/instance:
uuid: "fc0637683ceb456a85e28a28c089f769"
format_stamp: "Formatted at 2026-08-06 19:38:42 on dist-test-slave-ss4n"
server_key: "0c4917a65b16685e503cec0cd23f324b"
server_key_iv: "37e84c1b49c8e404ebe6df38d85e3540"
server_key_version: "encryptionkey@0"
I20260806 19:38:42.575171 24371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/wal/instance:
uuid: "fc0637683ceb456a85e28a28c089f769"
format_stamp: "Formatted at 2026-08-06 19:38:42 on dist-test-slave-ss4n"
server_key: "0c4917a65b16685e503cec0cd23f324b"
server_key_iv: "37e84c1b49c8e404ebe6df38d85e3540"
server_key_version: "encryptionkey@0"
I20260806 19:38:42.581805 24371 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 19:38:42.586130 24386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:42.588267 24371 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260806 19:38:42.588503 24371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/wal
uuid: "fc0637683ceb456a85e28a28c089f769"
format_stamp: "Formatted at 2026-08-06 19:38:42 on dist-test-slave-ss4n"
server_key: "0c4917a65b16685e503cec0cd23f324b"
server_key_iv: "37e84c1b49c8e404ebe6df38d85e3540"
server_key_version: "encryptionkey@0"
I20260806 19:38:42.588754 24371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:42.610707 24371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:42.616088 24371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:42.616446 24371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:42.642707 24371 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.61:36915
I20260806 19:38:42.642757 24438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.61:36915 every 8 connection(s)
I20260806 19:38:42.644416 24371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/data/info.pb
I20260806 19:38:42.648862 24439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:38:42.653854 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 24371
I20260806 19:38:42.654116 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-1/wal/instance
I20260806 19:38:42.654953 10340 external_mini_cluster.cc:1428] Setting key 26633d8c713c42747a16c626f8151861
I20260806 19:38:42.657923 24439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:38:42 dist-test-slave-ss4n krb5kdc[24283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045122, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.61@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.25.60@KRBTEST.COM; defaulting to no policy
I20260806 19:38:42.687232 24441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.660959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36390 (local address 127.10.25.62:37499)
0806 19:38:42.661615 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:42.661624 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:42.662660 (+ 1036us) server_negotiation.cc:407] Connection header received
0806 19:38:42.666298 (+ 3638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:42.666304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:42.666395 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:42.666515 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:42.668494 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:42.669912 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:42.671936 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:42.672764 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:42.672932 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:42.677413 (+ 4481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:42.677423 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:42.677431 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:42.677551 (+ 120us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:42.682676 (+ 5125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:42.683853 (+ 1177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:42.683861 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:42.683865 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:42.684050 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:42.684703 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:42.684708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:42.684710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:42.685063 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:42.685195 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:42.686362 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0806 19:38:42.686758 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":327,"threads_started":1}
I20260806 19:38:42.688061 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.661390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:37499 (local address 127.0.0.1:36390)
0806 19:38:42.662430 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.662509 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:42.665827 (+ 3318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:42.666878 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:42.666889 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:42.667535 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:42.668221 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:42.668242 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:42.670171 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:42.670177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:42.671713 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:42.671737 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:42.673106 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:42.673112 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:42.673366 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:42.674347 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:42.674417 (+ 70us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:42.677051 (+ 2634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:42.682964 (+ 5913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:42.682978 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:42.682996 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:42.683637 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:42.684225 (+ 588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:42.684230 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:42.684233 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:42.684545 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:42.685230 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:42.685237 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:42.685775 (+ 538us) client_negotiation.cc:769] Sending connection context
0806 19:38:42.686058 (+ 283us) client_negotiation.cc:240] Negotiation successful
0806 19:38:42.686498 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":360,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.60@KRBTEST.COM" created.
I20260806 19:38:42.690399 24439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:38:42 dist-test-slave-ss4n krb5kdc[24283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045122, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.61@KRBTEST.COM for kudu/127.10.25.61@KRBTEST.COM
I20260806 19:38:42.713011 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.691493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:36915 (local address 127.0.0.1:50528)
0806 19:38:42.691866 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.691899 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:42.692053 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:42.692975 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:42.692989 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:42.693673 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:42.693945 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:42.693962 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:42.695697 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:42.695705 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:42.697422 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:42.697446 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:42.698624 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:42.698639 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:42.698919 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:42.699766 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:42.699800 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:42.702745 (+ 2945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:42.709404 (+ 6659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:42.709409 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:42.709413 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:42.710012 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:42.710590 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:42.710594 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:42.710597 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:42.710772 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:42.711792 (+ 1020us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:42.711799 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:42.712012 (+ 213us) client_negotiation.cc:769] Sending connection context
0806 19:38:42.712370 (+ 358us) client_negotiation.cc:240] Negotiation successful
0806 19:38:42.712698 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260806 19:38:42.713656 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.691683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50528 (local address 127.10.25.61:36915)
0806 19:38:42.692460 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:42.692467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:42.692499 (+ 32us) server_negotiation.cc:407] Connection header received
0806 19:38:42.692560 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:42.692567 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:42.692776 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:42.692919 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:42.694261 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:42.695499 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:42.697706 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:42.698422 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:42.698551 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:42.703077 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:42.703105 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:42.703130 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:42.703184 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:42.709048 (+ 5864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:42.710220 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:42.710234 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:42.710241 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:42.710420 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:42.710944 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:42.710953 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:42.710959 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:42.711543 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:42.711798 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:42.712669 (+ 871us) server_negotiation.cc:299] Negotiation successful
0806 19:38:42.713027 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":393,"threads_started":1}
I20260806 19:38:42.717365 24439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:38:42.718936 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.718375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50200)
0806 19:38:42.718625 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.718764 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260806 19:38:42.719261 24387 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
I20260806 19:38:42.721156 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.720748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50210)
0806 19:38:42.720929 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.721027 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 19:38:42.721468 24439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:34529: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.60.keytab.
Entry for principal kudu/127.10.25.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.60.keytab.
I20260806 19:38:42.735460 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.60:34529
--webserver_interface=127.10.25.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.60.keytab
--principal=kudu/127.10.25.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:37499,127.10.25.61:36915,127.10.25.60:34529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:38:42.769168 24364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } attempt: 5
I20260806 19:38:42.770982 24439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } attempt: 1
I20260806 19:38:42.772500 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.771793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50224)
0806 19:38:42.772138 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.772276 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197}
I20260806 19:38:42.773789 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.773330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50232)
0806 19:38:42.773571 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.773657 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 19:38:42.774086 24439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:34529: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Aug 06 19:38:42 dist-test-slave-ss4n krb5kdc[24283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045121, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for kudu/127.10.25.61@KRBTEST.COM
I20260806 19:38:42.789388 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.770302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:36915 (local address 127.0.0.1:50536)
0806 19:38:42.770566 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.770594 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:42.770966 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:42.771914 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:42.771923 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:42.772414 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:42.772718 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:42.772736 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:42.774649 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:42.774655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:42.775703 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:42.775728 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:42.777336 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:42.777345 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:42.777623 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:42.778396 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:42.778418 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:42.781107 (+ 2689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:42.785731 (+ 4624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:42.785739 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:42.785743 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:42.786388 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:42.787027 (+ 639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:42.787031 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:42.787034 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:42.787394 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:42.788369 (+ 975us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:42.788374 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:42.788563 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 19:38:42.788783 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 19:38:42.789096 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260806 19:38:42.789585 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.770593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50536 (local address 127.10.25.61:36915)
0806 19:38:42.771056 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:42.771067 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:42.771095 (+ 28us) server_negotiation.cc:407] Connection header received
0806 19:38:42.771184 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:42.771190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:42.771591 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:42.771829 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:42.773009 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:42.774425 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:42.776008 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:42.777065 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:42.777236 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:42.781376 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:42.781383 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:42.781387 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:42.781432 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:42.785294 (+ 3862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:42.786610 (+ 1316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:42.786617 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:42.786621 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:42.786832 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:42.787689 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:42.787696 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:42.787699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:42.788197 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:42.788378 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:42.788928 (+ 550us) server_negotiation.cc:299] Negotiation successful
0806 19:38:42.789198 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229}
I20260806 19:38:42.790982 24364 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:38:42.792382 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.791887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50244)
0806 19:38:42.792112 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.792214 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 19:38:42.794109 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.793412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50246)
0806 19:38:42.793782 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.793948 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219}
W20260806 19:38:42.794552 24364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:34529: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
I20260806 19:38:42.842469 24439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } attempt: 2
I20260806 19:38:42.844198 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.843613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50262)
0806 19:38:42.843885 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.843998 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260806 19:38:42.845325 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.844940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50268)
0806 19:38:42.845111 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.845196 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 19:38:42.845582 24439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:34529: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
I20260806 19:38:42.845880 24364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } attempt: 1
I20260806 19:38:42.847043 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.846536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50284)
0806 19:38:42.846760 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.846871 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 19:38:42.848179 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.847797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50294)
0806 19:38:42.847975 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.848049 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260806 19:38:42.848414 24364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:34529: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
I20260806 19:38:42.916669 24364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } attempt: 2
I20260806 19:38:42.918123 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.917547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50310)
0806 19:38:42.917819 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.917924 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 19:38:42.919840 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.919221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50312)
0806 19:38:42.919525 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.919638 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260806 19:38:42.920229 24364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:34529: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
I20260806 19:38:42.936865 24439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } attempt: 3
I20260806 19:38:42.938268 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.937741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50318)
0806 19:38:42.937988 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.938090 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 19:38:42.939457 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:42.938964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50324)
0806 19:38:42.939229 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:42.939326 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260806 19:38:42.939704 24439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:34529: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
I20260806 19:38:43.026772 24364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } attempt: 3
I20260806 19:38:43.028596 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.028001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50334)
0806 19:38:43.028290 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:43.028392 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260806 19:38:43.029855 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.029395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50342)
0806 19:38:43.029650 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:43.029733 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260806 19:38:43.030088 24364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:34529: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
I20260806 19:38:43.083021 24439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } attempt: 4
I20260806 19:38:43.084721 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.084051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50356)
0806 19:38:43.084399 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:43.084514 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260806 19:38:43.086208 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.085758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50368)
0806 19:38:43.085966 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:43.086058 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260806 19:38:43.086467 24439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:34529: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
W20260806 19:38:43.166317 24447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:43.166728 24447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:43.166905 24447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:43.175319 24447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:38:43.175509 24447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:43.175627 24447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:43.175734 24447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:38:43.175817 24447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:38:43.186849 24447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:37499,127.10.25.61:36915,127.10.25.60:34529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.60.keytab
--principal=kudu/127.10.25.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.60:34529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.25.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:43.188418 24447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:43.189996 24447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:43.206141 24452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:43.206229 24453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:43.206470 24455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:43.207240 24447 server_base.cc:1034] running on GCE node
I20260806 19:38:43.207326 24364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } attempt: 4
I20260806 19:38:43.208482 24447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:43.208992 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.208317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50374)
0806 19:38:43.208648 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:43.208789 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260806 19:38:43.210358 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.209885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50376)
0806 19:38:43.210073 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:43.210173 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 19:38:43.210768 24364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:34529: Network error: Client connection negotiation failed: client connection to 127.10.25.60:34529: connect: Connection refused (error 111)
I20260806 19:38:43.210969 24447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:43.212376 24447 hybrid_clock.cc:648] HybridClock initialized: now 1786045123212321 us; error 184 us; skew 500 ppm
I20260806 19:38:43.212826 24447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:43 dist-test-slave-ss4n krb5kdc[24283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045123, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:43.216423 24447 init.cc:377] Logged in from keytab as kudu/127.10.25.60@KRBTEST.COM (short username kudu)
I20260806 19:38:43.218390 24447 webserver.cc:460] Webserver started at http://127.10.25.60:33255/ using document root <none> and password file <none>
I20260806 19:38:43.219161 24447 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:43.219296 24447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:43.219575 24447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:38:43.223064 24447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/data/instance:
uuid: "7f253b126986447480f02fc608c1d53f"
format_stamp: "Formatted at 2026-08-06 19:38:43 on dist-test-slave-ss4n"
server_key: "05c1bb5210a7b0f6e1d25d1c4fabadb1"
server_key_iv: "e821b2cfbb2dc1e494f763c8ccf11abd"
server_key_version: "encryptionkey@0"
I20260806 19:38:43.223990 24447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/wal/instance:
uuid: "7f253b126986447480f02fc608c1d53f"
format_stamp: "Formatted at 2026-08-06 19:38:43 on dist-test-slave-ss4n"
server_key: "05c1bb5210a7b0f6e1d25d1c4fabadb1"
server_key_iv: "e821b2cfbb2dc1e494f763c8ccf11abd"
server_key_version: "encryptionkey@0"
I20260806 19:38:43.231087 24447 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260806 19:38:43.236303 24462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:43.238826 24447 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260806 19:38:43.239111 24447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/wal
uuid: "7f253b126986447480f02fc608c1d53f"
format_stamp: "Formatted at 2026-08-06 19:38:43 on dist-test-slave-ss4n"
server_key: "05c1bb5210a7b0f6e1d25d1c4fabadb1"
server_key_iv: "e821b2cfbb2dc1e494f763c8ccf11abd"
server_key_version: "encryptionkey@0"
I20260806 19:38:43.239398 24447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:43.257658 24447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:43.264254 24447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:43.264627 24447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:43.292008 24447 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.60:34529
I20260806 19:38:43.292033 24515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.60:34529 every 8 connection(s)
I20260806 19:38:43.294420 24447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/data/info.pb
I20260806 19:38:43.295205 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 24447
I20260806 19:38:43.295537 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786044861763400-10340-0/minicluster-data/master-2/wal/instance
I20260806 19:38:43.296350 10340 external_mini_cluster.cc:1428] Setting key 2feb91783a8d9adccbf877366581879b
I20260806 19:38:43.300835 24516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:38:43.310873 24516 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:38:43 dist-test-slave-ss4n krb5kdc[24283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.60@KRBTEST.COM
Aug 06 19:38:43 dist-test-slave-ss4n krb5kdc[24283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045123, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.60@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:38:43.329959 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.299690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50380 (local address 127.10.25.60:34529)
0806 19:38:43.300969 (+ 1279us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:43.301110 (+ 141us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:43.301527 (+ 417us) server_negotiation.cc:407] Connection header received
0806 19:38:43.304151 (+ 2624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:43.304181 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:43.304790 (+ 609us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:43.305310 (+ 520us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:43.307236 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.309432 (+ 2196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.311190 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.312342 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.312555 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:43.318295 (+ 5740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:43.318308 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:43.318325 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:43.318377 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:43.322589 (+ 4212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.323757 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.323766 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.323774 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.323985 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.324736 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.324748 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.324754 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.326158 (+ 1404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:43.326593 (+ 435us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:43.327615 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0806 19:38:43.328370 (+ 755us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":958,"thread_start_us":563,"threads_started":1}
I20260806 19:38:43.335454 24521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.314338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:37499 (local address 127.0.0.1:36394)
0806 19:38:43.315387 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:43.315436 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:43.315658 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:43.316123 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:43.316131 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:43.316846 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:43.317265 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:43.317279 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.318844 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.318850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:43.320563 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:43.320585 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.321799 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.321805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:43.321971 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:43.322788 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:43.322815 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:43.326590 (+ 3775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:43.331143 (+ 4553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:43.331160 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:43.331169 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:43.331993 (+ 824us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:43.332653 (+ 660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:43.332658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:43.332662 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:43.332878 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:43.334042 (+ 1164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:43.334050 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:43.334308 (+ 258us) client_negotiation.cc:769] Sending connection context
0806 19:38:43.334527 (+ 219us) client_negotiation.cc:240] Negotiation successful
0806 19:38:43.334941 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"thread_start_us":632,"threads_started":1}
I20260806 19:38:43.335639 24520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.314314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36394 (local address 127.10.25.62:37499)
0806 19:38:43.315015 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:43.315024 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:43.315546 (+ 522us) server_negotiation.cc:407] Connection header received
0806 19:38:43.315828 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:43.315832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:43.315903 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:43.316041 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:43.317467 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.318552 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.320750 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.321547 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.321704 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:43.326917 (+ 5213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:43.326924 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:43.326928 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:43.326977 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:43.330734 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.332254 (+ 1520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.332261 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.332264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.332483 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.333141 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.333151 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.333154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.333819 (+ 665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:43.333977 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:43.334727 (+ 750us) server_negotiation.cc:299] Negotiation successful
0806 19:38:43.335041 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":368,"threads_started":1}
I20260806 19:38:43.337811 24516 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:38:43 dist-test-slave-ss4n krb5kdc[24283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
Aug 06 19:38:43 dist-test-slave-ss4n krb5kdc[24283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045123, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.60@KRBTEST.COM for kudu/127.10.25.61@KRBTEST.COM
I20260806 19:38:43.353803 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.335033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36406 (local address 127.10.25.62:37499)
0806 19:38:43.336146 (+ 1113us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:43.336167 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:43.336195 (+ 28us) server_negotiation.cc:407] Connection header received
0806 19:38:43.336259 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:43.336263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:43.336352 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:43.336483 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:43.338332 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.339749 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.341244 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.341786 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.341932 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:43.347526 (+ 5594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:43.347533 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:43.347537 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:43.347581 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:43.350855 (+ 3274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.351675 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.351682 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.351686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.351828 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.352226 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.352231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.352234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.352539 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:43.352845 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:43.353136 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 19:38:43.353376 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":895,"thread_start_us":634,"threads_started":1}
I20260806 19:38:43.355281 24521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.338788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:36915 (local address 127.0.0.1:50540)
0806 19:38:43.338992 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:43.339012 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:43.339130 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:43.340074 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:43.340078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:43.340520 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:43.340699 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:43.340708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.341933 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.341939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:43.343176 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:43.343190 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.344158 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.344164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:43.344408 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:43.345059 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:43.345079 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:43.347672 (+ 2593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:43.352032 (+ 4360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:43.352039 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:43.352043 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:43.352554 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:43.353021 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:43.353027 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:43.353031 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:43.353219 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:43.354118 (+ 899us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:43.354123 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:43.354335 (+ 212us) client_negotiation.cc:769] Sending connection context
0806 19:38:43.354532 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 19:38:43.354916 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 19:38:43.355410 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.338963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50540 (local address 127.10.25.61:36915)
0806 19:38:43.339553 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:43.339560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:43.339579 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:38:43.339656 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:43.339663 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:43.339753 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:43.339874 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:43.340893 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.341776 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.343397 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.343968 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.344135 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:43.347925 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:43.347931 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:43.347935 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:43.347974 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:43.351809 (+ 3835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.352716 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.352723 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.352727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.352886 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.353352 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.353358 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.353361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.353820 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:43.354047 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:43.354644 (+ 597us) server_negotiation.cc:299] Negotiation successful
0806 19:38:43.354832 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":336,"threads_started":1}
I20260806 19:38:43.356420 24516 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:38:43 dist-test-slave-ss4n krb5kdc[24283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045123, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.60@KRBTEST.COM for kudu/127.10.25.60@KRBTEST.COM
Aug 06 19:38:43 dist-test-slave-ss4n krb5kdc[24283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.61@KRBTEST.COM
I20260806 19:38:43.373984 24521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.357201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50390)
0806 19:38:43.357402 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:43.357426 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:43.357578 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:43.357955 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:43.357961 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:43.358463 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:43.358620 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:43.358628 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.359981 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.359987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:43.361620 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:43.361638 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.362498 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.362504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:43.362769 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:43.363608 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:43.363636 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:43.367184 (+ 3548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:43.371221 (+ 4037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:43.371225 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:43.371228 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:43.371633 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:43.372030 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:43.372036 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:43.372040 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:43.372175 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:43.372723 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:43.372728 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:43.372888 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 19:38:43.373048 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 19:38:43.373440 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":1}
I20260806 19:38:43.374011 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.357535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50390 (local address 127.10.25.60:34529)
0806 19:38:43.357689 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:43.357694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:43.357710 (+ 16us) server_negotiation.cc:407] Connection header received
0806 19:38:43.357749 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:43.357752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:43.357807 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:43.357889 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:43.358771 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.359833 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.361798 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.362312 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.362478 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:43.367433 (+ 4955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:43.367439 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:43.367443 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:43.367524 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:43.370827 (+ 3303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.371776 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.371782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.371785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.371917 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.372293 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.372298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.372301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.372607 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:43.372769 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:43.373141 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 19:38:43.373659 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":201,"server-negotiator.queue_time_us":36}
I20260806 19:38:43.377208 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.356099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50556 (local address 127.10.25.61:36915)
0806 19:38:43.356312 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:43.356317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:43.356337 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:38:43.356389 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:43.356396 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:43.356481 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:43.356595 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:43.358317 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.359453 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.361629 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.362563 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.362813 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:43.368839 (+ 6026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:43.368846 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:43.368849 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:43.368894 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:43.372681 (+ 3787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.373924 (+ 1243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.373930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.373933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.374137 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.375108 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.375113 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.375116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.375557 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:43.376074 (+ 517us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:43.376513 (+ 439us) server_negotiation.cc:299] Negotiation successful
0806 19:38:43.376824 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 19:38:43.381048 24516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f: Bootstrap starting.
I20260806 19:38:43.384977 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:38:43.385790 24516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f: Neither blocks nor log segments found. Creating new log.
I20260806 19:38:43.385705 24439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } attempt: 5
I20260806 19:38:43.387288 24516 log.cc:824] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:43.392495 24516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f: No bootstrap required, opened a new log
Aug 06 19:38:43 dist-test-slave-ss4n krb5kdc[24283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045122, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.61@KRBTEST.COM for kudu/127.10.25.60@KRBTEST.COM
I20260806 19:38:43.410456 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.386434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50400)
0806 19:38:43.386638 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:43.386661 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:43.386817 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:43.389418 (+ 2601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:43.389427 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:43.390223 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:43.390491 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:43.390504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.391902 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.391912 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:43.393319 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:43.393344 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.396730 (+ 3386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.396737 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:43.396980 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:43.397980 (+ 1000us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:43.398008 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:43.401362 (+ 3354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:43.406707 (+ 5345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:43.406714 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:43.406718 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:43.407409 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:43.407969 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:43.407975 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:43.407980 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:43.408194 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:43.409159 (+ 965us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:43.409165 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:43.409344 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 19:38:43.409546 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 19:38:43.409923 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 19:38:43.412235 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.388369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50400 (local address 127.10.25.60:34529)
0806 19:38:43.388739 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:43.388747 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:43.388834 (+ 87us) server_negotiation.cc:407] Connection header received
0806 19:38:43.388937 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:43.388943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:43.389056 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:43.390675 (+ 1619us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:43.390838 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.391713 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.393616 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.394595 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.394856 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:43.401666 (+ 6810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:43.401674 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:43.401679 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:43.401733 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:43.406460 (+ 4727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.407595 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.407603 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.407608 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.407795 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.408347 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.408352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.408358 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.408850 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:43.409091 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:43.411456 (+ 2365us) server_negotiation.cc:299] Negotiation successful
0806 19:38:43.411775 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260806 19:38:43.417320 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.394687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36414 (local address 127.10.25.62:37499)
0806 19:38:43.395441 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:43.395448 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:43.395475 (+ 27us) server_negotiation.cc:407] Connection header received
0806 19:38:43.395580 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:43.395585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:43.395690 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:43.395887 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:43.398430 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.399569 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.402301 (+ 2732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.403250 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.403422 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:43.407684 (+ 4262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:43.407692 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:43.407697 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:43.407749 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:43.412348 (+ 4599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.413870 (+ 1522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.413880 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.413886 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.414107 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.414575 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.414581 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.414585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.414993 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:43.415286 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:43.416666 (+ 1380us) server_negotiation.cc:299] Negotiation successful
0806 19:38:43.416956 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516}
I20260806 19:38:43.418669 24516 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } }
I20260806 19:38:43.419344 24516 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:38:43.419572 24516 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f253b126986447480f02fc608c1d53f, State: Initialized, Role: FOLLOWER
I20260806 19:38:43.420415 24516 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } }
I20260806 19:38:43.421262 24439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769: Bootstrap starting.
I20260806 19:38:43.421746 24533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } } }
I20260806 19:38:43.422494 24533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:38:43.423211 24516 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:38:43.425680 24439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769: Neither blocks nor log segments found. Creating new log.
I20260806 19:38:43.426924 24439 log.cc:824] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:43.428964 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.396282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50562 (local address 127.10.25.61:36915)
0806 19:38:43.396536 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:43.396542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:43.396564 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:38:43.401961 (+ 5397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:43.401967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:43.402068 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:43.402145 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:43.405318 (+ 3173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.406707 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.409117 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.409895 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.410010 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:43.414369 (+ 4359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:43.414378 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:43.414382 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:43.414429 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:43.418778 (+ 4349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.420915 (+ 2137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.420924 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.420927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.421157 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.423477 (+ 2320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.423483 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.423486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.424031 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:43.424310 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:43.427802 (+ 3492us) server_negotiation.cc:299] Negotiation successful
0806 19:38:43.428215 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260806 19:38:43.438503 24439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769: No bootstrap required, opened a new log
I20260806 19:38:43.443020 24531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.395739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50406 (local address 127.10.25.60:34529)
0806 19:38:43.396687 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:43.396694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:43.416445 (+ 19751us) server_negotiation.cc:407] Connection header received
0806 19:38:43.418203 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:43.418209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:43.418300 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:43.418391 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:43.421729 (+ 3338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.423246 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.426956 (+ 3710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.427749 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.427957 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:43.433039 (+ 5082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:43.433050 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:43.433058 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:43.433128 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:43.438776 (+ 5648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.440035 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.440041 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.440044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.440232 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.440799 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.440805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.440809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.441210 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:43.441553 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:43.442218 (+ 665us) server_negotiation.cc:299] Negotiation successful
0806 19:38:43.442507 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":778,"thread_start_us":426,"threads_started":1}
I20260806 19:38:43.449651 24439 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } }
I20260806 19:38:43.450148 24439 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:38:43.450301 24439 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc0637683ceb456a85e28a28c089f769, State: Initialized, Role: FOLLOWER
I20260806 19:38:43.451186 24439 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } }
W20260806 19:38:43.456915 24545 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:38:43.457180 24545 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:38:43.457017 24533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:38:43.457521 24533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } }
I20260806 19:38:43.458374 24547 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } } }
I20260806 19:38:43.458811 24547 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:38:43.459571 24533 leader_election.cc:288] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cba0c6d82d243d18914c2e323698041 (127.10.25.62:37499), fc0637683ceb456a85e28a28c089f769 (127.10.25.61:36915)
I20260806 19:38:43.460806 24439 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:38:43.462674 24414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f253b126986447480f02fc608c1d53f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0637683ceb456a85e28a28c089f769" is_pre_election: true
I20260806 19:38:43.463420 24414 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f253b126986447480f02fc608c1d53f in term 0.
I20260806 19:38:43.464299 24466 leader_election.cc:302] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f253b126986447480f02fc608c1d53f, fc0637683ceb456a85e28a28c089f769; no voters:
I20260806 19:38:43.467685 24533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:38:43.467955 24533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:38:43.468109 24533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:38:43.451013 24447 master_runner.cc:186] Error getting master registration for 127.10.25.62:37499: OK, instance_id { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" instance_seqno: 1786045122041643 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 19:38:43.470669 24447 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 19:38:43.472787 24533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } }
I20260806 19:38:43.474495 24414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f253b126986447480f02fc608c1d53f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc0637683ceb456a85e28a28c089f769"
I20260806 19:38:43.474786 24414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:38:43.478993 24414 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f253b126986447480f02fc608c1d53f in term 1.
I20260806 19:38:43.479522 24466 leader_election.cc:302] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f253b126986447480f02fc608c1d53f, fc0637683ceb456a85e28a28c089f769; no voters:
I20260806 19:38:43.479542 24533 leader_election.cc:288] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [CANDIDATE]: Term 1 election: Requested vote from peers 1cba0c6d82d243d18914c2e323698041 (127.10.25.62:37499), fc0637683ceb456a85e28a28c089f769 (127.10.25.61:36915)
I20260806 19:38:43.480654 24533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:38:43.481395 24533 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [term 1 LEADER]: Becoming Leader. State: Replica: 7f253b126986447480f02fc608c1d53f, State: Running, Role: LEADER
I20260806 19:38:43.488113 24364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } attempt: 5
I20260806 19:38:43.483820 24533 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } }
Aug 06 19:38:43 dist-test-slave-ss4n krb5kdc[24283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045121, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for kudu/127.10.25.60@KRBTEST.COM
I20260806 19:38:43.513691 24551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f253b126986447480f02fc608c1d53f. Latest consensus state: current_term: 1 leader_uuid: "7f253b126986447480f02fc608c1d53f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } } }
I20260806 19:38:43.514470 24551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [sys.catalog]: This master's current role is: LEADER
I20260806 19:38:43.517326 24560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:38:43.517379 24367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.488864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:34529 (local address 127.0.0.1:50412)
0806 19:38:43.489316 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:43.489340 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:43.489461 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:43.490610 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:43.490615 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:43.491413 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:43.491695 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:43.491709 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.493853 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.493866 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:43.495749 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:43.495776 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.497677 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.497684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:43.498027 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:43.499082 (+ 1055us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:43.499110 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:43.503860 (+ 4750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:43.511991 (+ 8131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:43.512000 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:43.512007 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:43.512699 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:43.514339 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:43.514346 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:43.514350 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:43.514601 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:43.516079 (+ 1478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:43.516084 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:43.516268 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 19:38:43.516414 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 19:38:43.516960 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301}
W20260806 19:38:43.517879 24468 tablet.cc:2369] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:38:43.520784 24560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:38:43.521399 24531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:43.489820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50412 (local address 127.10.25.60:34529)
0806 19:38:43.490022 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:43.490030 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:43.490051 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:38:43.490133 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:43.490138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:43.490262 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:43.490369 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:43.491920 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.493314 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.496348 (+ 3034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:43.497104 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:43.497231 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:43.504257 (+ 7026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:43.504265 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:43.504271 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:43.504325 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:43.511700 (+ 7375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.512898 (+ 1198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.512903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.512907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.514067 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:43.515013 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:43.515018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:43.515021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:43.515844 (+ 823us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:43.518359 (+ 2515us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:43.518679 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 19:38:43.519751 (+ 1072us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 19:38:43.530432 24364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041: Bootstrap starting.
I20260806 19:38:43.535802 24364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041: Neither blocks nor log segments found. Creating new log.
I20260806 19:38:43.543658 24364 log.cc:824] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:43.549417 24364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041: No bootstrap required, opened a new log
I20260806 19:38:43.541337 24371 master_runner.cc:186] Error getting master registration for 127.10.25.62:37499: OK, instance_id { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" instance_seqno: 1786045122041643 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 19:38:43.553637 24414 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769 [term 1 FOLLOWER]: Refusing update from remote peer 7f253b126986447480f02fc608c1d53f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:38:43.562779 24551 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:38:43.564893 24364 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } }
I20260806 19:38:43.565434 24364 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:38:43.565629 24364 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cba0c6d82d243d18914c2e323698041, State: Initialized, Role: FOLLOWER
I20260806 19:38:43.566586 24364 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } }
W20260806 19:38:43.558445 24562 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:38:43.569809 24562 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:38:43.571743 24371 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 19:38:43.576064 24547 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f253b126986447480f02fc608c1d53f. Latest consensus state: current_term: 1 leader_uuid: "7f253b126986447480f02fc608c1d53f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } } }
I20260806 19:38:43.576222 24566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } } }
I20260806 19:38:43.576668 24566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:38:43.576481 24547 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:38:43.578783 24338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f253b126986447480f02fc608c1d53f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cba0c6d82d243d18914c2e323698041" is_pre_election: true
I20260806 19:38:43.578850 24337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f253b126986447480f02fc608c1d53f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cba0c6d82d243d18914c2e323698041"
I20260806 19:38:43.578765 24364 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:38:43.579515 24336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:38:43.587561 24336 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041 [term 1 FOLLOWER]: Refusing update from remote peer 7f253b126986447480f02fc608c1d53f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:38:43.589316 24551 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:38:43.604193 24564 mvcc.cc:205] Tried to move back new op lower bound from 7315640826063360000 to 7315640825824067584. Current Snapshot: MvccSnapshot[applied={T|T < 7315640826063360000}]
I20260806 19:38:43.627856 24547 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7f253b126986447480f02fc608c1d53f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } } }
I20260806 19:38:43.628183 24547 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:38:43.629971 24566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f253b126986447480f02fc608c1d53f. Latest consensus state: current_term: 1 leader_uuid: "7f253b126986447480f02fc608c1d53f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } } }
I20260806 19:38:43.630448 24566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:38:43.638690 24560 catalog_manager.cc:1347] Generated new cluster ID: 1b512c6a39aa45f394bdd9447cc25a6b
I20260806 19:38:43.638849 24560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:38:43.639858 24571 mvcc.cc:205] Tried to move back new op lower bound from 7315640826063360000 to 7315640825824067584. Current Snapshot: MvccSnapshot[applied={T|T < 7315640826063360000}]
I20260806 19:38:43.639770 24533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7f253b126986447480f02fc608c1d53f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } } }
I20260806 19:38:43.640223 24533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [sys.catalog]: This master's current role is: LEADER
I20260806 19:38:43.640471 24533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7f253b126986447480f02fc608c1d53f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } } }
I20260806 19:38:43.640777 24533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f [sys.catalog]: This master's current role is: LEADER
I20260806 19:38:43.642686 24566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7f253b126986447480f02fc608c1d53f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cba0c6d82d243d18914c2e323698041" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 37499 } } peers { permanent_uuid: "fc0637683ceb456a85e28a28c089f769" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 36915 } } peers { permanent_uuid: "7f253b126986447480f02fc608c1d53f" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 34529 } } }
I20260806 19:38:43.643150 24566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:38:43.688015 24560 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:38:43.690338 24560 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:38:43.700284 24586 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 19:38:43.720407 24560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f: Generated new TSK 0
I20260806 19:38:43.722792 24560 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:38:44.572579 24562 catalog_manager.cc:1259] Loaded cluster ID: 1b512c6a39aa45f394bdd9447cc25a6b
I20260806 19:38:44.572695 24562 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769: loading cluster ID for follower catalog manager: success
I20260806 19:38:44.575268 24562 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769: acquiring CA information for follower catalog manager: success
I20260806 19:38:44.577208 24562 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fc0637683ceb456a85e28a28c089f769: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 19:38:44.708036 24586 catalog_manager.cc:1259] Loaded cluster ID: 1b512c6a39aa45f394bdd9447cc25a6b
I20260806 19:38:44.708179 24586 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041: loading cluster ID for follower catalog manager: success
I20260806 19:38:44.711634 24586 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041: acquiring CA information for follower catalog manager: success
I20260806 19:38:44.714280 24586 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1cba0c6d82d243d18914c2e323698041: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 19:38:44.840492 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 24296
I20260806 19:38:44.842648 24359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:44.954502 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 24296
I20260806 19:38:44.979808 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 24371
I20260806 19:38:44.981130 24434 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:45.101406 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 24371
I20260806 19:38:45.119583 24595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:45.118730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:37499 (local address 127.0.0.1:36426)
0806 19:38:45.119277 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:45.119407 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:37499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":318,"threads_started":1}
W20260806 19:38:45.119870 24463 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.25.62:37499: connect: Connection refused (error 111)
I20260806 19:38:45.124812 24595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:45.124334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:37499 (local address 127.0.0.1:36428)
0806 19:38:45.124524 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:45.124678 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:37499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 19:38:45.125033 24463 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7f253b126986447480f02fc608c1d53f -> Peer 1cba0c6d82d243d18914c2e323698041 (127.10.25.62:37499): Couldn't send request to peer 1cba0c6d82d243d18914c2e323698041. Status: Network error: Client connection negotiation failed: client connection to 127.10.25.62:37499: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 19:38:45.126966 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 24447
I20260806 19:38:45.128172 24510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:45.251403 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 24447
2026-08-06T19:38:45Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5294 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:38:45 dist-test-slave-ss4n krb5kdc[24600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:38:45 dist-test-slave-ss4n krb5kdc[24600](info): set up 2 sockets
Aug 06 19:38:45 dist-test-slave-ss4n krb5kdc[24600](info): commencing operation
krb5kdc: starting...
W20260806 19:38:46.484299 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:38:46 dist-test-slave-ss4n krb5kdc[24600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:38:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:38:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:38:46.819437 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:40569
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:40849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:40569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:38:47.216807 24614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:47.217144 24614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:47.217304 24614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:47.225733 24614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:38:47.225886 24614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:47.225936 24614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:47.226071 24614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:38:47.226172 24614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:38:47.237162 24614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:40849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:40569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:40569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:47.239276 24614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:47.241505 24614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:47.257050 24620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:47.257076 24622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:47.257197 24619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:47.258164 24614 server_base.cc:1034] running on GCE node
I20260806 19:38:47.259037 24614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:47.260829 24614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:47.262110 24614 hybrid_clock.cc:648] HybridClock initialized: now 1786045127262026 us; error 69 us; skew 500 ppm
I20260806 19:38:47.262607 24614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:47 dist-test-slave-ss4n krb5kdc[24600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045127, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:47.267498 24614 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:38:47.269562 24614 webserver.cc:460] Webserver started at http://127.10.25.62:39427/ using document root <none> and password file <none>
I20260806 19:38:47.270383 24614 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:47.270618 24614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:47.271064 24614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:38:47.274719 24614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "8ba7a14dddcc429ebc8019918e26ded6"
format_stamp: "Formatted at 2026-08-06 19:38:47 on dist-test-slave-ss4n"
server_key: "40087b93369ea7838436fa662ad577c9"
server_key_iv: "7c56ab2a75544984e0abfa4f3d70c01f"
server_key_version: "encryptionkey@0"
I20260806 19:38:47.275658 24614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "8ba7a14dddcc429ebc8019918e26ded6"
format_stamp: "Formatted at 2026-08-06 19:38:47 on dist-test-slave-ss4n"
server_key: "40087b93369ea7838436fa662ad577c9"
server_key_iv: "7c56ab2a75544984e0abfa4f3d70c01f"
server_key_version: "encryptionkey@0"
I20260806 19:38:47.282536 24614 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 19:38:47.288491 24629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:47.290565 24614 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 19:38:47.290819 24614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "8ba7a14dddcc429ebc8019918e26ded6"
format_stamp: "Formatted at 2026-08-06 19:38:47 on dist-test-slave-ss4n"
server_key: "40087b93369ea7838436fa662ad577c9"
server_key_iv: "7c56ab2a75544984e0abfa4f3d70c01f"
server_key_version: "encryptionkey@0"
I20260806 19:38:47.291082 24614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:47.339529 24614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:47.345937 24614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:47.346411 24614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:47.369813 24614 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:40569
I20260806 19:38:47.369859 24682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:40569 every 8 connection(s)
I20260806 19:38:47.371431 24614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:38:47.376096 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 24614
I20260806 19:38:47.376271 24683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:38:47.376399 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:38:47.377286 10340 external_mini_cluster.cc:1428] Setting key 6a2251b91cb48da9ae1cd04c00ff5de3
I20260806 19:38:47.395725 24683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6: Bootstrap starting.
I20260806 19:38:47.399458 24683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6: Neither blocks nor log segments found. Creating new log.
Aug 06 19:38:47 dist-test-slave-ss4n krb5kdc[24600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:38:47.400823 24683 log.cc:824] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:47.404579 24683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6: No bootstrap required, opened a new log
I20260806 19:38:47.412712 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:47.380173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45852 (local address 127.10.25.62:40569)
0806 19:38:47.381555 (+ 1382us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:47.381579 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:47.381647 (+ 68us) server_negotiation.cc:407] Connection header received
0806 19:38:47.384800 (+ 3153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:47.384860 (+ 60us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:47.385838 (+ 978us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:47.386423 (+ 585us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:47.389033 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:47.390742 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:47.393127 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:47.394174 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:47.394359 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:47.401904 (+ 7545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:47.401955 (+ 51us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:47.401995 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:47.402107 (+ 112us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:47.405978 (+ 3871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:47.407380 (+ 1402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:47.407400 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:47.407411 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:47.407832 (+ 421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:47.408485 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:47.408501 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:47.408504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:47.409660 (+ 1156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:47.410168 (+ 508us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:47.410831 (+ 663us) server_negotiation.cc:299] Negotiation successful
0806 19:38:47.411352 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1032,"thread_start_us":681,"threads_started":1}
I20260806 19:38:47.414252 24683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba7a14dddcc429ebc8019918e26ded6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40569 } }
I20260806 19:38:47.414698 24683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:38:47.414821 24683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ba7a14dddcc429ebc8019918e26ded6, State: Initialized, Role: FOLLOWER
I20260806 19:38:47.415432 24683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba7a14dddcc429ebc8019918e26ded6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40569 } }
I20260806 19:38:47.415731 24683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:38:47.415880 24683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:38:47.416065 24683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:38:47.418390 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:38:47.419019 24683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba7a14dddcc429ebc8019918e26ded6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40569 } }
I20260806 19:38:47.419701 24683 leader_election.cc:302] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ba7a14dddcc429ebc8019918e26ded6; no voters:
I20260806 19:38:47.420446 24683 leader_election.cc:288] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:38:47.420637 24688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:38:47.421360 24688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [term 1 LEADER]: Becoming Leader. State: Replica: 8ba7a14dddcc429ebc8019918e26ded6, State: Running, Role: LEADER
I20260806 19:38:47.422137 24688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba7a14dddcc429ebc8019918e26ded6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40569 } }
I20260806 19:38:47.424440 24683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:38:47.425733 24690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ba7a14dddcc429ebc8019918e26ded6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba7a14dddcc429ebc8019918e26ded6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40569 } } }
I20260806 19:38:47.425685 24689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ba7a14dddcc429ebc8019918e26ded6. Latest consensus state: current_term: 1 leader_uuid: "8ba7a14dddcc429ebc8019918e26ded6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba7a14dddcc429ebc8019918e26ded6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40569 } } }
I20260806 19:38:47.426367 24689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [sys.catalog]: This master's current role is: LEADER
I20260806 19:38:47.426429 24690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6 [sys.catalog]: This master's current role is: LEADER
I20260806 19:38:47.430794 24693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:38:47.437795 24693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:38:47.448626 24693 catalog_manager.cc:1347] Generated new cluster ID: 680c4b7415054dc0bde8e902409ed0e5
I20260806 19:38:47.448794 24693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:38:47.480882 24693 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:38:47.482476 24693 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:38:47.494477 24693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ba7a14dddcc429ebc8019918e26ded6: Generated new TSK 0
I20260806 19:38:47.496080 24693 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:38:47.535631 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 24614
I20260806 19:38:47.537163 24678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:47.641214 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 24614
2026-08-06T19:38:47Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2377 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:38:47 dist-test-slave-ss4n krb5kdc[24712](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:38:47 dist-test-slave-ss4n krb5kdc[24712](info): set up 2 sockets
Aug 06 19:38:47 dist-test-slave-ss4n krb5kdc[24712](info): commencing operation
krb5kdc: starting...
W20260806 19:38:48.904484 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:38:49 dist-test-slave-ss4n krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:38:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:38:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:38:49.218983 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:36513
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:36513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:38:49.588340 24725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:49.588702 24725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:49.588856 24725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:49.596632 24725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:38:49.596778 24725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:49.596868 24725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:49.596966 24725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:38:49.597034 24725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:38:49.607767 24725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:36513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:36513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:49.609321 24725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:49.610860 24725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:49.627233 24730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:49.627210 24731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:49.627184 24733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:49.628365 24725 server_base.cc:1034] running on GCE node
I20260806 19:38:49.629287 24725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:49.631135 24725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:49.632457 24725 hybrid_clock.cc:648] HybridClock initialized: now 1786045129632406 us; error 51 us; skew 500 ppm
I20260806 19:38:49.632917 24725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:49 dist-test-slave-ss4n krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045129, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:49.636598 24725 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:38:49.638710 24725 webserver.cc:460] Webserver started at http://127.10.25.62:41781/ using document root <none> and password file <none>
I20260806 19:38:49.639523 24725 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:49.639667 24725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:49.639940 24725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:38:49.643848 24725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "30875fa4eb10456d9b4873efd27524a9"
format_stamp: "Formatted at 2026-08-06 19:38:49 on dist-test-slave-ss4n"
server_key: "077204bc75284e6ad342f177201aba24"
server_key_iv: "d7eb87a401d837b0159bd6573ccc4faa"
server_key_version: "encryptionkey@0"
I20260806 19:38:49.645041 24725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "30875fa4eb10456d9b4873efd27524a9"
format_stamp: "Formatted at 2026-08-06 19:38:49 on dist-test-slave-ss4n"
server_key: "077204bc75284e6ad342f177201aba24"
server_key_iv: "d7eb87a401d837b0159bd6573ccc4faa"
server_key_version: "encryptionkey@0"
I20260806 19:38:49.651501 24725 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 19:38:49.656589 24740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:49.658721 24725 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 19:38:49.658955 24725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "30875fa4eb10456d9b4873efd27524a9"
format_stamp: "Formatted at 2026-08-06 19:38:49 on dist-test-slave-ss4n"
server_key: "077204bc75284e6ad342f177201aba24"
server_key_iv: "d7eb87a401d837b0159bd6573ccc4faa"
server_key_version: "encryptionkey@0"
I20260806 19:38:49.659188 24725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:49.675974 24725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:49.681725 24725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:49.682060 24725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:49.707299 24725 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:36513
I20260806 19:38:49.707314 24793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:36513 every 8 connection(s)
I20260806 19:38:49.708909 24725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:38:49.713418 24794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:38:49.713420 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 24725
I20260806 19:38:49.713758 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:38:49.714567 10340 external_mini_cluster.cc:1428] Setting key 2d582e965f026440f968db5d0a30900e
I20260806 19:38:49.728740 24794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9: Bootstrap starting.
I20260806 19:38:49.733590 24794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9: Neither blocks nor log segments found. Creating new log.
Aug 06 19:38:49 dist-test-slave-ss4n krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:38:49.735114 24794 log.cc:824] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:49.739966 24794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9: No bootstrap required, opened a new log
I20260806 19:38:49.747088 24797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:49.717030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50094 (local address 127.10.25.62:36513)
0806 19:38:49.718412 (+ 1382us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:49.718434 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:49.718514 (+ 80us) server_negotiation.cc:407] Connection header received
0806 19:38:49.721228 (+ 2714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:49.721268 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:49.721963 (+ 695us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:49.722492 (+ 529us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:49.724987 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:49.726757 (+ 1770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:49.728385 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:49.729547 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:49.729758 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:49.736142 (+ 6384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:49.736163 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:49.736193 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:49.736277 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:49.740701 (+ 4424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:49.741993 (+ 1292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:49.742006 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:49.742017 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:49.742476 (+ 459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:49.743141 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:49.743147 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:49.743159 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:49.743942 (+ 783us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:49.744252 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:49.745042 (+ 790us) server_negotiation.cc:299] Negotiation successful
0806 19:38:49.745827 (+ 785us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":979,"thread_start_us":620,"threads_started":1}
I20260806 19:38:49.750627 24794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } }
I20260806 19:38:49.751183 24794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:38:49.751344 24794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30875fa4eb10456d9b4873efd27524a9, State: Initialized, Role: FOLLOWER
I20260806 19:38:49.752060 24794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } }
I20260806 19:38:49.752348 24794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:38:49.752524 24794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:38:49.752736 24794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:38:49.755507 24794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } }
I20260806 19:38:49.756250 24794 leader_election.cc:302] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30875fa4eb10456d9b4873efd27524a9; no voters:
I20260806 19:38:49.757032 24794 leader_election.cc:288] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:38:49.757231 24799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:38:49.758185 24799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 1 LEADER]: Becoming Leader. State: Replica: 30875fa4eb10456d9b4873efd27524a9, State: Running, Role: LEADER
I20260806 19:38:49.758848 24799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } }
I20260806 19:38:49.759680 24794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:38:49.763072 24800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30875fa4eb10456d9b4873efd27524a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } } }
I20260806 19:38:49.763157 24801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30875fa4eb10456d9b4873efd27524a9. Latest consensus state: current_term: 1 leader_uuid: "30875fa4eb10456d9b4873efd27524a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } } }
I20260806 19:38:49.764101 24801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [sys.catalog]: This master's current role is: LEADER
I20260806 19:38:49.764096 24800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [sys.catalog]: This master's current role is: LEADER
I20260806 19:38:49.772873 24806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:38:49.778352 24806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:38:49.788887 24806 catalog_manager.cc:1347] Generated new cluster ID: 9a9b65f399994c3fbf4522d93bbbe58a
I20260806 19:38:49.789058 24806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:38:49.827642 24806 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:38:49.829813 24806 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:38:49.839908 24806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9: Generated new TSK 0
I20260806 19:38:49.841248 24806 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:38:49.927078 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36513
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:38:50.321544 24821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:50.321905 24821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:50.322057 24821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:50.330762 24821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:50.330911 24821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:50.331099 24821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:38:50.343905 24821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:50.345427 24821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:50.347780 24821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:50.365438 24827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:50.365586 24826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:50.366099 24829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:50.366974 24821 server_base.cc:1034] running on GCE node
I20260806 19:38:50.367770 24821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:50.368973 24821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:50.370255 24821 hybrid_clock.cc:648] HybridClock initialized: now 1786045130370220 us; error 75 us; skew 500 ppm
I20260806 19:38:50.370692 24821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:50 dist-test-slave-ss4n krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045130, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:50.380511 24821 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:38:50.382751 24821 webserver.cc:460] Webserver started at http://127.10.25.1:45283/ using document root <none> and password file <none>
I20260806 19:38:50.383661 24821 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:50.383862 24821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:50.384258 24821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:38:50.387979 24821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "17dc696ed80d4e52beb51b7a6fa9cdac"
format_stamp: "Formatted at 2026-08-06 19:38:50 on dist-test-slave-ss4n"
server_key: "ef68a4bf0f4386e7d7c905718bc25e48"
server_key_iv: "f2ed245b745eb7109ce8199be75f4321"
server_key_version: "encryptionkey@0"
I20260806 19:38:50.389003 24821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "17dc696ed80d4e52beb51b7a6fa9cdac"
format_stamp: "Formatted at 2026-08-06 19:38:50 on dist-test-slave-ss4n"
server_key: "ef68a4bf0f4386e7d7c905718bc25e48"
server_key_iv: "f2ed245b745eb7109ce8199be75f4321"
server_key_version: "encryptionkey@0"
I20260806 19:38:50.396512 24821 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 19:38:50.402642 24836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:50.404842 24821 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:38:50.405097 24821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "17dc696ed80d4e52beb51b7a6fa9cdac"
format_stamp: "Formatted at 2026-08-06 19:38:50 on dist-test-slave-ss4n"
server_key: "ef68a4bf0f4386e7d7c905718bc25e48"
server_key_iv: "f2ed245b745eb7109ce8199be75f4321"
server_key_version: "encryptionkey@0"
I20260806 19:38:50.405361 24821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:50.427106 24821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:50.432708 24821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:50.433077 24821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:50.434276 24821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:38:50.437009 24821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:38:50.437165 24821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:50.437366 24821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:38:50.437448 24821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:50.480914 24821 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:34819
I20260806 19:38:50.480978 24950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:34819 every 8 connection(s)
I20260806 19:38:50.482507 24821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:38:50.483939 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 24821
I20260806 19:38:50.484200 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:38:50.485013 10340 external_mini_cluster.cc:1428] Setting key c5428e952569accdfde32f5ba1e87462
Aug 06 19:38:50 dist-test-slave-ss4n krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045130, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:38:50.510797 24954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:50.486121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:55543 (local address 127.10.25.62:36513)
0806 19:38:50.486704 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:50.486711 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:50.488216 (+ 1505us) server_negotiation.cc:407] Connection header received
0806 19:38:50.491015 (+ 2799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:50.491022 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:50.491154 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:50.491299 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:50.494384 (+ 3085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:50.495342 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:50.497368 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:50.497874 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:50.498035 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:50.502875 (+ 4840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:50.502882 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:50.502886 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:50.502938 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:50.506777 (+ 3839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:50.507826 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:50.507831 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:50.507834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:50.507982 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:50.508510 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:50.508514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:50.508516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:50.508825 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:50.509016 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:50.510085 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0806 19:38:50.510400 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":285,"threads_started":1}
I20260806 19:38:50.511853 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:50.486710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:55543)
0806 19:38:50.487768 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:50.487980 (+ 212us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:50.490542 (+ 2562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:50.491858 (+ 1316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:50.491871 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:50.492756 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:50.493982 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:50.494044 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:50.495617 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:50.495622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:50.497140 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:50.497158 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:50.498207 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:50.498216 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:50.498487 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:50.499284 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:50.499316 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:50.502375 (+ 3059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:50.507027 (+ 4652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:50.507045 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:50.507063 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:50.507644 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:50.508113 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:50.508118 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:50.508122 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:50.508351 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:50.508975 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:50.508996 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:50.509565 (+ 569us) client_negotiation.cc:769] Sending connection context
0806 19:38:50.509962 (+ 397us) client_negotiation.cc:240] Negotiation successful
0806 19:38:50.510602 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":392,"threads_started":1}
I20260806 19:38:50.513991 24951 heartbeater.cc:347] Connected to a master server at 127.10.25.62:36513
I20260806 19:38:50.514358 24951 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
I20260806 19:38:50.516010 24951 heartbeater.cc:511] Master 127.10.25.62:36513 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
I20260806 19:38:50.519749 24759 ts_manager.cc:194] Registered new tserver with Master: 17dc696ed80d4e52beb51b7a6fa9cdac (127.10.25.1:34819)
I20260806 19:38:50.521404 24759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:55543
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:38:50.562450 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36513
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:38:50.956130 24958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:50.956473 24958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:50.956621 24958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:50.965025 24958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:50.965185 24958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:50.965381 24958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:38:50.976037 24958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:50.977629 24958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:50.979157 24958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:50.995048 24963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:50.995014 24966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:50.995101 24964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:50.996790 24958 server_base.cc:1034] running on GCE node
I20260806 19:38:50.997692 24958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:50.998809 24958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:51.000070 24958 hybrid_clock.cc:648] HybridClock initialized: now 1786045130999993 us; error 79 us; skew 500 ppm
I20260806 19:38:51.000505 24958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:50 dist-test-slave-ss4n krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045130, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:51.004575 24958 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:38:51.006762 24958 webserver.cc:460] Webserver started at http://127.10.25.2:41079/ using document root <none> and password file <none>
I20260806 19:38:51.007596 24958 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:51.007766 24958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:51.008109 24958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:38:51.011655 24958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "90058d5c0bcb4f42a4b17d02706b5a6e"
format_stamp: "Formatted at 2026-08-06 19:38:51 on dist-test-slave-ss4n"
server_key: "1b57ea1441450e48bea23fdc9fccdd74"
server_key_iv: "412479fe73d4bac4ea075c0c59165739"
server_key_version: "encryptionkey@0"
I20260806 19:38:51.012612 24958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "90058d5c0bcb4f42a4b17d02706b5a6e"
format_stamp: "Formatted at 2026-08-06 19:38:51 on dist-test-slave-ss4n"
server_key: "1b57ea1441450e48bea23fdc9fccdd74"
server_key_iv: "412479fe73d4bac4ea075c0c59165739"
server_key_version: "encryptionkey@0"
I20260806 19:38:51.018998 24958 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260806 19:38:51.023556 24974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:51.025359 24958 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260806 19:38:51.025681 24958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "90058d5c0bcb4f42a4b17d02706b5a6e"
format_stamp: "Formatted at 2026-08-06 19:38:51 on dist-test-slave-ss4n"
server_key: "1b57ea1441450e48bea23fdc9fccdd74"
server_key_iv: "412479fe73d4bac4ea075c0c59165739"
server_key_version: "encryptionkey@0"
I20260806 19:38:51.025972 24958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:51.056021 24958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:51.061915 24958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:51.062292 24958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:51.063663 24958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:38:51.065528 24958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:38:51.065660 24958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 19:38:51.065840 24958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:38:51.065920 24958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:51.107403 24958 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:45813
I20260806 19:38:51.107461 25089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:45813 every 8 connection(s)
I20260806 19:38:51.108944 24958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:38:51.116969 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 24958
I20260806 19:38:51.117209 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:38:51.117795 10340 external_mini_cluster.cc:1428] Setting key 317dc03e6b6f2462948815f6b5e6f75e
Aug 06 19:38:51 dist-test-slave-ss4n krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045130, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:38:51.134315 25092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:51.112296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:42655 (local address 127.10.25.62:36513)
0806 19:38:51.113006 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:51.113011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:51.114315 (+ 1304us) server_negotiation.cc:407] Connection header received
0806 19:38:51.116562 (+ 2247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:51.116568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:51.116681 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:51.116832 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:51.118981 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.119845 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.121859 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.122398 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.122566 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:51.127536 (+ 4970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:51.127542 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:51.127545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:51.127587 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:51.130669 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:51.131656 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:51.131666 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:51.131669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:51.131840 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:51.132355 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:51.132360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:51.132364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:51.132683 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:51.132853 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:51.133685 (+ 832us) server_negotiation.cc:299] Negotiation successful
0806 19:38:51.133940 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":442,"threads_started":1}
I20260806 19:38:51.135421 25093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:51.113049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:42655)
0806 19:38:51.114090 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:51.114138 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:51.116176 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:51.117186 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:51.117196 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:51.118041 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:51.118752 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:51.118796 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.120060 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.120067 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:51.121666 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:51.121683 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.122712 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.122721 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:51.122995 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:51.123837 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:51.123866 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:51.127175 (+ 3309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:51.130894 (+ 3719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:51.130906 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:51.130936 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:51.131521 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:51.131980 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:51.131985 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:51.131989 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:51.132219 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:51.132832 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:51.132841 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:51.133230 (+ 389us) client_negotiation.cc:769] Sending connection context
0806 19:38:51.133546 (+ 316us) client_negotiation.cc:240] Negotiation successful
0806 19:38:51.134168 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":495,"threads_started":1}
I20260806 19:38:51.137233 25090 heartbeater.cc:347] Connected to a master server at 127.10.25.62:36513
I20260806 19:38:51.137663 25090 heartbeater.cc:464] Registering TS with master...
I20260806 19:38:51.138689 25090 heartbeater.cc:511] Master 127.10.25.62:36513 requested a full tablet report, sending...
I20260806 19:38:51.140679 24759 ts_manager.cc:194] Registered new tserver with Master: 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813)
I20260806 19:38:51.142150 24759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:42655
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:38:51.191200 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36513
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:38:51.525832 24951 heartbeater.cc:503] Master 127.10.25.62:36513 was elected leader, sending a full tablet report...
W20260806 19:38:51.580359 25096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:51.580704 25096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:51.580849 25096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:51.588663 25096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:51.588864 25096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:51.589062 25096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:38:51.599716 25096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:51.601183 25096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:51.602696 25096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:51.619728 25105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:51.619855 25102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:51.619896 25103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:51.621356 25096 server_base.cc:1034] running on GCE node
I20260806 19:38:51.622679 25096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:51.624432 25096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:51.625769 25096 hybrid_clock.cc:648] HybridClock initialized: now 1786045131625715 us; error 56 us; skew 500 ppm
I20260806 19:38:51.626227 25096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:51 dist-test-slave-ss4n krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045131, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:51.630910 25096 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:38:51.632882 25096 webserver.cc:460] Webserver started at http://127.10.25.3:42037/ using document root <none> and password file <none>
I20260806 19:38:51.633911 25096 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:51.634099 25096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:51.634608 25096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:38:51.638257 25096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "b8da86666b7d4778807c51842118d38c"
format_stamp: "Formatted at 2026-08-06 19:38:51 on dist-test-slave-ss4n"
server_key: "5aa9ac2e3b535f9d54aae9d6c00584f2"
server_key_iv: "9509ed937a51c329027c0afdac872da6"
server_key_version: "encryptionkey@0"
I20260806 19:38:51.639256 25096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "b8da86666b7d4778807c51842118d38c"
format_stamp: "Formatted at 2026-08-06 19:38:51 on dist-test-slave-ss4n"
server_key: "5aa9ac2e3b535f9d54aae9d6c00584f2"
server_key_iv: "9509ed937a51c329027c0afdac872da6"
server_key_version: "encryptionkey@0"
I20260806 19:38:51.647890 25096 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.004s
I20260806 19:38:51.652844 25112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:51.654841 25096 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 19:38:51.655071 25096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "b8da86666b7d4778807c51842118d38c"
format_stamp: "Formatted at 2026-08-06 19:38:51 on dist-test-slave-ss4n"
server_key: "5aa9ac2e3b535f9d54aae9d6c00584f2"
server_key_iv: "9509ed937a51c329027c0afdac872da6"
server_key_version: "encryptionkey@0"
I20260806 19:38:51.655328 25096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:51.674917 25096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:51.679864 25096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:51.680208 25096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:51.681326 25096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:38:51.683090 25096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:38:51.683207 25096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:51.683374 25096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:38:51.683450 25096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:51.727782 25096 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:37663
I20260806 19:38:51.727908 25227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:37663 every 8 connection(s)
I20260806 19:38:51.729212 25096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:38:51.736011 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 25096
I20260806 19:38:51.736230 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:38:51.736830 10340 external_mini_cluster.cc:1428] Setting key 70838604117975b77e80c3fcea2faed8
Aug 06 19:38:51 dist-test-slave-ss4n krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045131, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:38:51.756700 25230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:51.733082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:48961 (local address 127.10.25.62:36513)
0806 19:38:51.733770 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:51.733774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:51.734899 (+ 1125us) server_negotiation.cc:407] Connection header received
0806 19:38:51.737697 (+ 2798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:51.737703 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:51.737810 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:51.737941 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:51.739894 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.740754 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.742608 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.743044 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.743191 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:51.747828 (+ 4637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:51.747833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:51.747837 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:51.747877 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:51.751426 (+ 3549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:51.752825 (+ 1399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:51.752833 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:51.752838 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:51.753132 (+ 294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:51.754408 (+ 1276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:51.754415 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:51.754419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:51.754935 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:51.755130 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:51.755959 (+ 829us) server_negotiation.cc:299] Negotiation successful
0806 19:38:51.756286 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":287,"threads_started":1}
I20260806 19:38:51.758282 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:51.733828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:48961)
0806 19:38:51.734690 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:51.734742 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:51.737238 (+ 2496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:51.738242 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:51.738252 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:51.739028 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:51.739701 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:51.739721 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.740956 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.740960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:51.742411 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:51.742427 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.743315 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.743323 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:51.743583 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:51.744736 (+ 1153us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:51.744767 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:51.747503 (+ 2736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:51.751678 (+ 4175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:51.751698 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:51.751717 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:51.752512 (+ 795us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:51.753352 (+ 840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:51.753356 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:51.753359 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:51.754245 (+ 886us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:51.755087 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:51.755097 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:51.755541 (+ 444us) client_negotiation.cc:769] Sending connection context
0806 19:38:51.755853 (+ 312us) client_negotiation.cc:240] Negotiation successful
0806 19:38:51.756479 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":378,"threads_started":1}
I20260806 19:38:51.760300 25228 heartbeater.cc:347] Connected to a master server at 127.10.25.62:36513
I20260806 19:38:51.760656 25228 heartbeater.cc:464] Registering TS with master...
I20260806 19:38:51.761658 25228 heartbeater.cc:511] Master 127.10.25.62:36513 requested a full tablet report, sending...
I20260806 19:38:51.763880 24759 ts_manager.cc:194] Registered new tserver with Master: b8da86666b7d4778807c51842118d38c (127.10.25.3:37663)
I20260806 19:38:51.765596 24759 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:48961
I20260806 19:38:51.769238 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:38:51.794051 25230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:51.775529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50096 (local address 127.10.25.62:36513)
0806 19:38:51.775991 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:51.776001 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:51.776529 (+ 528us) server_negotiation.cc:407] Connection header received
0806 19:38:51.776863 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:51.776870 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:51.777161 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:51.777599 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:51.780627 (+ 3028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.781601 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.783303 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.783807 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.783982 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:51.786922 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:51.786928 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:51.786932 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:51.786981 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:51.790742 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:51.791671 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:51.791679 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:51.791683 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:51.791847 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:51.792274 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:51.792280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:51.792284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:51.792670 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:51.792964 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:51.793341 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 19:38:51.793672 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260806 19:38:51.800158 24759 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:38:51.809144 24759 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:38:51.851349 25241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:51.826293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:45813 (local address 127.0.0.1:49214)
0806 19:38:51.828036 (+ 1743us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:51.828096 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:51.828355 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:51.842649 (+ 14294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:51.842681 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:51.842724 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:51.843231 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:51.843246 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.845831 (+ 2585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.845837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:51.848528 (+ 2691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:51.848549 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.850196 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.850202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:51.850418 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:51.850468 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 19:38:51.850643 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 19:38:51.850867 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1359,"mutex_wait_us":74,"thread_start_us":1136,"threads_started":1}
I20260806 19:38:51.851524 25244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:51.826964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49214 (local address 127.10.25.2:45813)
0806 19:38:51.841710 (+ 14746us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:51.841722 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:51.841770 (+ 48us) server_negotiation.cc:407] Connection header received
0806 19:38:51.841919 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:51.841930 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:51.842319 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:51.842564 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:51.843925 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.845670 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.848751 (+ 3081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.849955 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.850137 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:51.850379 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:51.850949 (+ 570us) server_negotiation.cc:299] Negotiation successful
0806 19:38:51.851159 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14558,"thread_start_us":14399,"threads_started":1}
I20260806 19:38:51.851915 25245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:51.828715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:37663 (local address 127.0.0.1:48866)
0806 19:38:51.842806 (+ 14091us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:51.842826 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:51.842987 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:51.843662 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:51.843666 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:51.843688 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:51.844184 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:51.844199 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.846128 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.846136 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:51.848746 (+ 2610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:51.848761 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.850989 (+ 2228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.850996 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:51.851205 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:51.851238 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 19:38:51.851343 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 19:38:51.851483 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13959,"mutex_wait_us":67,"thread_start_us":14294,"threads_started":1}
I20260806 19:38:51.852388 25243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:51.826418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34684 (local address 127.10.25.1:34819)
0806 19:38:51.841251 (+ 14833us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:51.841264 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:51.842385 (+ 1121us) server_negotiation.cc:407] Connection header received
0806 19:38:51.842599 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:51.842608 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:51.842847 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:51.843045 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:51.844240 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.845357 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.848805 (+ 3448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.850441 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.850601 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:51.850815 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:51.851174 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 19:38:51.851370 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14584,"thread_start_us":14391,"threads_started":1}
I20260806 19:38:51.852979 25246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:51.829591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48866 (local address 127.10.25.3:37663)
0806 19:38:51.837178 (+ 7587us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:51.837193 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:51.842970 (+ 5777us) server_negotiation.cc:407] Connection header received
0806 19:38:51.843218 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:51.843228 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:51.843441 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:51.843666 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:51.844403 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.845969 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.848952 (+ 2983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.850467 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.850591 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:51.850815 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:51.852434 (+ 1619us) server_negotiation.cc:299] Negotiation successful
0806 19:38:51.852610 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7309,"thread_start_us":6664,"threads_started":1}
I20260806 19:38:51.853658 25242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:51.826327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:34819 (local address 127.0.0.1:34684)
0806 19:38:51.842163 (+ 15836us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:51.842210 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:51.842405 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:51.843344 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:51.843348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:51.843373 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:51.843968 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:51.843983 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.845542 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.845548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:51.848591 (+ 3043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:51.848607 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.850676 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.850682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:51.850835 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:51.850867 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 19:38:51.850965 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 19:38:51.853259 (+ 2294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15697,"mutex_wait_us":2123,"thread_start_us":15516,"threads_started":1}
I20260806 19:38:51.856201 25023 tablet_service.cc:1467] Processing CreateTablet for tablet dce295007add4268abfade2348abeffc (DEFAULT_TABLE table=test-table [id=be9585f5b62c4bb6bf080a33f43365a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:38:51.857880 25023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dce295007add4268abfade2348abeffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:38:51.857340 24885 tablet_service.cc:1467] Processing CreateTablet for tablet dce295007add4268abfade2348abeffc (DEFAULT_TABLE table=test-table [id=be9585f5b62c4bb6bf080a33f43365a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:38:51.858868 24885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dce295007add4268abfade2348abeffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:38:51.860248 25161 tablet_service.cc:1467] Processing CreateTablet for tablet dce295007add4268abfade2348abeffc (DEFAULT_TABLE table=test-table [id=be9585f5b62c4bb6bf080a33f43365a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:38:51.861863 25161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dce295007add4268abfade2348abeffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:38:51.874763 25248 tablet_bootstrap.cc:492] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e: Bootstrap starting.
I20260806 19:38:51.875949 25247 tablet_bootstrap.cc:492] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac: Bootstrap starting.
I20260806 19:38:51.878001 25249 tablet_bootstrap.cc:492] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c: Bootstrap starting.
I20260806 19:38:51.881321 25247 tablet_bootstrap.cc:654] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac: Neither blocks nor log segments found. Creating new log.
I20260806 19:38:51.881546 25249 tablet_bootstrap.cc:654] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c: Neither blocks nor log segments found. Creating new log.
I20260806 19:38:51.882323 25248 tablet_bootstrap.cc:654] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e: Neither blocks nor log segments found. Creating new log.
I20260806 19:38:51.882985 25249 log.cc:824] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:51.883375 25247 log.cc:824] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:51.883667 25248 log.cc:824] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:51.887228 25247 tablet_bootstrap.cc:492] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac: No bootstrap required, opened a new log
I20260806 19:38:51.887524 25247 ts_tablet_manager.cc:1397] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260806 19:38:51.888407 25249 tablet_bootstrap.cc:492] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c: No bootstrap required, opened a new log
I20260806 19:38:51.888772 25249 ts_tablet_manager.cc:1397] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260806 19:38:51.890350 25248 tablet_bootstrap.cc:492] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e: No bootstrap required, opened a new log
I20260806 19:38:51.890825 25248 ts_tablet_manager.cc:1397] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.000s
I20260806 19:38:51.898159 25249 raft_consensus.cc:348] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:51.898702 25249 raft_consensus.cc:374] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:38:51.898818 25249 raft_consensus.cc:729] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8da86666b7d4778807c51842118d38c, State: Initialized, Role: FOLLOWER
I20260806 19:38:51.899611 25249 consensus_queue.cc:260] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:51.900300 25247 raft_consensus.cc:348] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:51.900826 25247 raft_consensus.cc:374] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:38:51.900935 25247 raft_consensus.cc:729] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17dc696ed80d4e52beb51b7a6fa9cdac, State: Initialized, Role: FOLLOWER
I20260806 19:38:51.901122 25228 heartbeater.cc:503] Master 127.10.25.62:36513 was elected leader, sending a full tablet report...
I20260806 19:38:51.901795 25247 consensus_queue.cc:260] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:51.902735 25249 ts_tablet_manager.cc:1428] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260806 19:38:51.902379 25248 raft_consensus.cc:348] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:51.903016 25248 raft_consensus.cc:374] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:38:51.903147 25248 raft_consensus.cc:729] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90058d5c0bcb4f42a4b17d02706b5a6e, State: Initialized, Role: FOLLOWER
I20260806 19:38:51.903908 25248 consensus_queue.cc:260] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:51.906526 25247 ts_tablet_manager.cc:1428] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac: Time spent starting tablet: real 0.019s user 0.008s sys 0.009s
I20260806 19:38:51.907711 25090 heartbeater.cc:503] Master 127.10.25.62:36513 was elected leader, sending a full tablet report...
I20260806 19:38:51.909586 25248 ts_tablet_manager.cc:1428] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e: Time spent starting tablet: real 0.019s user 0.019s sys 0.000s
I20260806 19:38:51.978026 25253 raft_consensus.cc:482] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:38:51.978389 25253 raft_consensus.cc:504] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:51.980330 25253 leader_election.cc:288] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17dc696ed80d4e52beb51b7a6fa9cdac (127.10.25.1:34819), 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813)
W20260806 19:38:51.981640 25229 tablet.cc:2369] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:38:51.985818 24952 tablet.cc:2369] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:38:51.988257 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:51.980897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:45813 (local address 127.10.25.3:44769)
0806 19:38:51.981143 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:51.981166 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:51.981303 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:51.982190 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:51.982197 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:51.982244 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:51.982547 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:51.982566 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.983798 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.983808 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:51.985432 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:51.985447 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.987459 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.987465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:51.987679 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:51.987707 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 19:38:51.987802 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 19:38:51.987910 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 19:38:51.988444 25244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:51.981242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:44769 (local address 127.10.25.2:45813)
0806 19:38:51.981544 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:51.981554 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:51.981582 (+ 28us) server_negotiation.cc:407] Connection header received
0806 19:38:51.981685 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:51.981693 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:51.981794 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:51.981936 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:51.982788 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.983636 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.985755 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.987214 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.987405 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:51.987546 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:51.987890 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 19:38:51.988092 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 19:38:51.988948 25243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:51.980884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:56959 (local address 127.10.25.1:34819)
0806 19:38:51.981188 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:51.981196 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:51.982672 (+ 1476us) server_negotiation.cc:407] Connection header received
0806 19:38:51.982820 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:51.982830 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:51.982891 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:51.983017 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:51.983531 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.984247 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.986380 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.987912 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.988089 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:51.988212 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:51.988611 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 19:38:51.988767 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260806 19:38:51.989768 25256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:51.980914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:34819 (local address 127.10.25.3:56959)
0806 19:38:51.982527 (+ 1613us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:51.982544 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:51.982693 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:51.983031 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:51.983035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:51.983059 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:51.983361 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:51.983368 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.984412 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.984417 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:51.986135 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:51.986153 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:51.988198 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:51.988203 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:51.988401 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:51.988432 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 19:38:51.989161 (+ 729us) client_negotiation.cc:240] Negotiation successful
0806 19:38:51.989331 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1476,"mutex_wait_us":489,"thread_start_us":747,"threads_started":1}
I20260806 19:38:51.989794 25044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dce295007add4268abfade2348abeffc" candidate_uuid: "b8da86666b7d4778807c51842118d38c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" is_pre_election: true
I20260806 19:38:51.990427 25044 raft_consensus.cc:2454] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8da86666b7d4778807c51842118d38c in term 0.
I20260806 19:38:51.991056 24906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dce295007add4268abfade2348abeffc" candidate_uuid: "b8da86666b7d4778807c51842118d38c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" is_pre_election: true
I20260806 19:38:51.991473 25117 leader_election.cc:302] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90058d5c0bcb4f42a4b17d02706b5a6e, b8da86666b7d4778807c51842118d38c; no voters:
I20260806 19:38:51.991662 24906 raft_consensus.cc:2454] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8da86666b7d4778807c51842118d38c in term 0.
I20260806 19:38:51.992138 25253 raft_consensus.cc:2781] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:38:51.992363 25253 raft_consensus.cc:482] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:38:51.992543 25253 raft_consensus.cc:3037] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:38:51.995585 25253 raft_consensus.cc:504] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:51.996560 25253 leader_election.cc:288] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [CANDIDATE]: Term 1 election: Requested vote from peers 17dc696ed80d4e52beb51b7a6fa9cdac (127.10.25.1:34819), 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813)
I20260806 19:38:51.997371 25044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dce295007add4268abfade2348abeffc" candidate_uuid: "b8da86666b7d4778807c51842118d38c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e"
I20260806 19:38:51.997527 24906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dce295007add4268abfade2348abeffc" candidate_uuid: "b8da86666b7d4778807c51842118d38c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac"
I20260806 19:38:51.997756 25044 raft_consensus.cc:3037] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:38:51.997931 24906 raft_consensus.cc:3037] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:38:52.001436 24906 raft_consensus.cc:2454] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8da86666b7d4778807c51842118d38c in term 1.
I20260806 19:38:52.001613 25044 raft_consensus.cc:2454] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8da86666b7d4778807c51842118d38c in term 1.
I20260806 19:38:52.002187 25115 leader_election.cc:302] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17dc696ed80d4e52beb51b7a6fa9cdac, b8da86666b7d4778807c51842118d38c; no voters:
I20260806 19:38:52.002594 25253 raft_consensus.cc:2781] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:38:52.003095 25253 raft_consensus.cc:686] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [term 1 LEADER]: Becoming Leader. State: Replica: b8da86666b7d4778807c51842118d38c, State: Running, Role: LEADER
I20260806 19:38:52.003633 25253 consensus_queue.cc:237] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:52.009408 24759 catalog_manager.cc:5495] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c reported cstate change: term changed from 0 to 1, leader changed from <none> to b8da86666b7d4778807c51842118d38c (127.10.25.3). New cstate: current_term: 1 leader_uuid: "b8da86666b7d4778807c51842118d38c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } health_report { overall_health: HEALTHY } } }
I20260806 19:38:52.054926 25246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:52.047312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34982 (local address 127.10.25.3:37663)
0806 19:38:52.047621 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:52.047627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:52.047654 (+ 27us) server_negotiation.cc:407] Connection header received
0806 19:38:52.047866 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:52.047871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:52.047962 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:52.048138 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:38:52.048642 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:52.049461 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:52.051321 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:52.052066 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:52.052493 (+ 427us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:52.053131 (+ 638us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:38:52.053866 (+ 735us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:38:52.054018 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:52.054184 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 19:38:52.054277 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260806 19:38:52.066021 25044 raft_consensus.cc:1261] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e [term 1 FOLLOWER]: Refusing update from remote peer b8da86666b7d4778807c51842118d38c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:38:52.066036 24906 raft_consensus.cc:1261] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Refusing update from remote peer b8da86666b7d4778807c51842118d38c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:38:52.067241 25253 consensus_queue.cc:1035] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [LEADER]: Connected to new peer: Peer: permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:38:52.067610 25257 consensus_queue.cc:1035] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [LEADER]: Connected to new peer: Peer: permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:38:52.103931 24757 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:38:52.104285 24757 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:38:52.109597 24757 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9: Sending DeleteTablet for 3 replicas of tablet dce295007add4268abfade2348abeffc
I20260806 19:38:52.111202 24885 tablet_service.cc:1514] Processing DeleteTablet for tablet dce295007add4268abfade2348abeffc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:38:52 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:34684
I20260806 19:38:52.111541 25023 tablet_service.cc:1514] Processing DeleteTablet for tablet dce295007add4268abfade2348abeffc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:38:52 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:49214
I20260806 19:38:52.111671 25161 tablet_service.cc:1514] Processing DeleteTablet for tablet dce295007add4268abfade2348abeffc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:38:52 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:48866
I20260806 19:38:52.113027 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 24821
I20260806 19:38:52.128139 25273 tablet_replica.cc:330] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c: stopping tablet replica
I20260806 19:38:52.129164 25273 raft_consensus.cc:2229] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:38:52.130102 25273 raft_consensus.cc:2258] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:38:52.132838 25273 ts_tablet_manager.cc:1905] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:38:52.135591 25271 tablet_replica.cc:330] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac: stopping tablet replica
I20260806 19:38:52.136379 25271 raft_consensus.cc:2229] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:38:52.138712 25272 tablet_replica.cc:330] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e: stopping tablet replica
I20260806 19:38:52.139292 25272 raft_consensus.cc:2229] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:38:52.141588 25273 ts_tablet_manager.cc:1918] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:38:52.141889 25273 log.cc:1196] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/dce295007add4268abfade2348abeffc
I20260806 19:38:52.142645 25273 ts_tablet_manager.cc:1939] T dce295007add4268abfade2348abeffc P b8da86666b7d4778807c51842118d38c: Deleting consensus metadata
Aug 06 19:38:52 dist-test-slave-ss4n krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:38:52.144342 25272 raft_consensus.cc:2258] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:38:52.144896 24744 catalog_manager.cc:4841] TS b8da86666b7d4778807c51842118d38c (127.10.25.3:37663): tablet dce295007add4268abfade2348abeffc (table test-table [id=be9585f5b62c4bb6bf080a33f43365a2]) successfully deleted
I20260806 19:38:52.146096 25271 raft_consensus.cc:2258] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:38:52.148522 25271 ts_tablet_manager.cc:1905] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:38:52.148531 25272 ts_tablet_manager.cc:1905] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:38:52.157963 25271 ts_tablet_manager.cc:1918] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:38:52.158154 25272 ts_tablet_manager.cc:1918] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:38:52.158334 25271 log.cc:1196] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/dce295007add4268abfade2348abeffc
I20260806 19:38:52.158409 25272 log.cc:1196] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/dce295007add4268abfade2348abeffc
I20260806 19:38:52.159094 25272 ts_tablet_manager.cc:1939] T dce295007add4268abfade2348abeffc P 90058d5c0bcb4f42a4b17d02706b5a6e: Deleting consensus metadata
I20260806 19:38:52.159171 25271 ts_tablet_manager.cc:1939] T dce295007add4268abfade2348abeffc P 17dc696ed80d4e52beb51b7a6fa9cdac: Deleting consensus metadata
I20260806 19:38:52.161645 24742 catalog_manager.cc:4841] TS 17dc696ed80d4e52beb51b7a6fa9cdac (127.10.25.1:34819): tablet dce295007add4268abfade2348abeffc (table test-table [id=be9585f5b62c4bb6bf080a33f43365a2]) successfully deleted
I20260806 19:38:52.161645 24744 catalog_manager.cc:4841] TS 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813): tablet dce295007add4268abfade2348abeffc (table test-table [id=be9585f5b62c4bb6bf080a33f43365a2]) successfully deleted
I20260806 19:38:52.164335 25243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:52.114535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41486 (local address 127.10.25.1:34819)
0806 19:38:52.114778 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:52.114785 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:52.121424 (+ 6639us) server_negotiation.cc:407] Connection header received
0806 19:38:52.123044 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:52.123052 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:52.123150 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:52.123228 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:52.127876 (+ 4648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:52.129205 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:52.135756 (+ 6551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:52.136609 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:52.136752 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:52.154474 (+ 17722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:52.154497 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:52.154518 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:52.154590 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:52.159482 (+ 4892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:52.160891 (+ 1409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:52.160903 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:52.160907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:52.161118 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:52.161765 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:52.161770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:52.161774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:52.162292 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:52.162658 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:52.163540 (+ 882us) server_negotiation.cc:299] Negotiation successful
0806 19:38:52.163832 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 19:38:52.165187 24946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:52.264957 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 24821
I20260806 19:38:52.288318 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 24958
Aug 06 19:38:52 dist-test-slave-ss4n krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:38:52.310793 25244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:52.289571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48106 (local address 127.10.25.2:45813)
0806 19:38:52.289969 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:52.289979 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:52.290062 (+ 83us) server_negotiation.cc:407] Connection header received
0806 19:38:52.290318 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:52.290332 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:52.290433 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:52.290543 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:52.292462 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:52.293904 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:52.295997 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:52.296425 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:52.296559 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:52.302965 (+ 6406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:52.302989 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:52.303007 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:52.303067 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:52.306827 (+ 3760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:52.307929 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:52.307940 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:52.307942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:52.308117 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:52.308653 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:52.308660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:52.308666 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:52.309188 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:52.309611 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:52.310050 (+ 439us) server_negotiation.cc:299] Negotiation successful
0806 19:38:52.310327 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260806 19:38:52.311668 25085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:52.412823 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 24958
I20260806 19:38:52.443809 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 25096
Aug 06 19:38:52 dist-test-slave-ss4n krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:38:52.465454 25246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:52.444802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34986 (local address 127.10.25.3:37663)
0806 19:38:52.444995 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:52.444999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:52.445081 (+ 82us) server_negotiation.cc:407] Connection header received
0806 19:38:52.445306 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:52.445311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:52.445385 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:52.445592 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:52.447817 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:52.448694 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:52.450836 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:52.451335 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:52.451467 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:52.457908 (+ 6441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:52.457926 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:52.457958 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:52.458005 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:52.461931 (+ 3926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:52.462851 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:52.462871 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:52.462873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:52.463046 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:52.463558 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:52.463562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:52.463564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:52.463981 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:52.464343 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:52.464677 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 19:38:52.464930 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 19:38:52.466272 25223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:52.601090 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 25096
I20260806 19:38:52.624833 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 24725
I20260806 19:38:52.626408 24789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:52.747074 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 24725
I20260806 19:38:52.778090 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:36513
--webserver_interface=127.10.25.62
--webserver_port=41781
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:36513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:38:53.166447 25279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:53.166913 25279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:53.167068 25279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:53.177460 25279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:38:53.177706 25279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:53.177822 25279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:53.177956 25279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:38:53.178012 25279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:38:53.190042 25279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:36513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:36513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=41781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:53.191743 25279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:53.193287 25279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:53.222909 25286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:53.222931 25285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:53.222851 25289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:53.227572 25279 server_base.cc:1034] running on GCE node
I20260806 19:38:53.228770 25279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:53.230875 25279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:53.232189 25279 hybrid_clock.cc:648] HybridClock initialized: now 1786045133232139 us; error 79 us; skew 500 ppm
I20260806 19:38:53.232685 25279 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:53 dist-test-slave-ss4n krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045133, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:53.237210 25279 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:38:53.239625 25279 webserver.cc:460] Webserver started at http://127.10.25.62:41781/ using document root <none> and password file <none>
I20260806 19:38:53.240583 25279 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:53.240785 25279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:53.248261 25279 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:38:53.252236 25296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:53.254524 25279 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 19:38:53.254796 25279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "30875fa4eb10456d9b4873efd27524a9"
format_stamp: "Formatted at 2026-08-06 19:38:49 on dist-test-slave-ss4n"
server_key: "077204bc75284e6ad342f177201aba24"
server_key_iv: "d7eb87a401d837b0159bd6573ccc4faa"
server_key_version: "encryptionkey@0"
I20260806 19:38:53.256099 25279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:53.275244 25279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:53.281366 25279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:53.281778 25279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:53.308801 25279 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:36513
I20260806 19:38:53.308821 25349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:36513 every 8 connection(s)
I20260806 19:38:53.310721 25279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:38:53.314298 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 25279
I20260806 19:38:53.315225 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:34819
--local_ip_for_outbound_sockets=127.10.25.1
--tserver_master_addrs=127.10.25.62:36513
--webserver_port=45283
--webserver_interface=127.10.25.1
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:38:53.322208 25350 sys_catalog.cc:263] Verifying existing consensus state
I20260806 19:38:53.326479 25350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9: Bootstrap starting.
I20260806 19:38:53.352336 25350 log.cc:824] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:53.366475 25350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:38:53.367110 25350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9: Bootstrap complete.
I20260806 19:38:53.377383 25350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } }
I20260806 19:38:53.378489 25350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30875fa4eb10456d9b4873efd27524a9, State: Initialized, Role: FOLLOWER
I20260806 19:38:53.379248 25350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } }
I20260806 19:38:53.379541 25350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:38:53.379691 25350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:38:53.379882 25350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 1 FOLLOWER]: Advancing to term 2
I20260806 19:38:53.384246 25350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } }
I20260806 19:38:53.384876 25350 leader_election.cc:302] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30875fa4eb10456d9b4873efd27524a9; no voters:
I20260806 19:38:53.385658 25350 leader_election.cc:288] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 19:38:53.385819 25354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 19:38:53.387192 25354 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 2 LEADER]: Becoming Leader. State: Replica: 30875fa4eb10456d9b4873efd27524a9, State: Running, Role: LEADER
I20260806 19:38:53.387931 25354 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } }
I20260806 19:38:53.388296 25350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:38:53.391076 25356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30875fa4eb10456d9b4873efd27524a9. Latest consensus state: current_term: 2 leader_uuid: "30875fa4eb10456d9b4873efd27524a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } } }
I20260806 19:38:53.391523 25355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "30875fa4eb10456d9b4873efd27524a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } } }
I20260806 19:38:53.392529 25356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [sys.catalog]: This master's current role is: LEADER
I20260806 19:38:53.392530 25355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [sys.catalog]: This master's current role is: LEADER
I20260806 19:38:53.394547 25360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:38:53.407146 25360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:38:53.409878 25360 catalog_manager.cc:1259] Loaded cluster ID: 9a9b65f399994c3fbf4522d93bbbe58a
I20260806 19:38:53.410276 25360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:38:53.415694 25360 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 19:38:53.443049 25360 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 19:38:53.458305 25360 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:38:53.458508 25374 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 19:38:53.460536 25360 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9: Loaded TSK: 0
I20260806 19:38:53.462095 25360 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 19:38:53.758677 25352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:53.759001 25352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:53.759156 25352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:53.767776 25352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:53.767911 25352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:53.768105 25352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:38:53.780792 25352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:34819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=45283
--tserver_master_addrs=127.10.25.62:36513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:53.782604 25352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:53.784626 25352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:53.807771 25379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:53.809468 25381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:53.810894 25383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:53.813033 25352 server_base.cc:1034] running on GCE node
I20260806 19:38:53.814046 25352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:53.815788 25352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:53.817212 25352 hybrid_clock.cc:648] HybridClock initialized: now 1786045133817133 us; error 71 us; skew 500 ppm
I20260806 19:38:53.817859 25352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:53 dist-test-slave-ss4n krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045133, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:53.823637 25352 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:38:53.826251 25352 webserver.cc:460] Webserver started at http://127.10.25.1:45283/ using document root <none> and password file <none>
I20260806 19:38:53.827605 25352 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:53.827877 25352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:53.837518 25352 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:38:53.842523 25390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:53.844734 25352 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 19:38:53.844988 25352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "17dc696ed80d4e52beb51b7a6fa9cdac"
format_stamp: "Formatted at 2026-08-06 19:38:50 on dist-test-slave-ss4n"
server_key: "ef68a4bf0f4386e7d7c905718bc25e48"
server_key_iv: "f2ed245b745eb7109ce8199be75f4321"
server_key_version: "encryptionkey@0"
I20260806 19:38:53.846768 25352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:53.875466 25352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:53.882306 25352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:53.882676 25352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:53.884140 25352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:38:53.886281 25352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:38:53.886422 25352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:53.886551 25352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:38:53.886641 25352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:53.940454 25352 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:34819
I20260806 19:38:53.940588 25507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:34819 every 8 connection(s)
I20260806 19:38:53.942269 25352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:38:53.951402 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 25352
I20260806 19:38:53.952383 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:45813
--local_ip_for_outbound_sockets=127.10.25.2
--tserver_master_addrs=127.10.25.62:36513
--webserver_port=41079
--webserver_interface=127.10.25.2
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:38:53 dist-test-slave-ss4n krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045133, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:38:53.975492 25510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:53.945707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:35563 (local address 127.10.25.62:36513)
0806 19:38:53.946957 (+ 1250us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:53.946974 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:53.947523 (+ 549us) server_negotiation.cc:407] Connection header received
0806 19:38:53.950218 (+ 2695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:53.950234 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:53.950840 (+ 606us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:53.951434 (+ 594us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:53.954055 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:53.956503 (+ 2448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:53.958899 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:53.959946 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:53.960227 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:53.965207 (+ 4980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:53.965217 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:53.965235 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:53.965296 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:53.969579 (+ 4283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:53.970669 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:53.970677 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:53.970684 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:53.970886 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:53.971562 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:53.971567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:53.971569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:53.972166 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:53.972621 (+ 455us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:53.973574 (+ 953us) server_negotiation.cc:299] Negotiation successful
0806 19:38:53.974060 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":378,"threads_started":1}
I20260806 19:38:53.975564 25511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:53.946095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:35563)
0806 19:38:53.947227 (+ 1132us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:53.947287 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:53.949372 (+ 2085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:53.951692 (+ 2320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:53.951709 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:53.952358 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:53.953286 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:53.953313 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:53.956845 (+ 3532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:53.956851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:53.958651 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:53.958683 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:53.960262 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:53.960272 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:53.960829 (+ 557us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:53.961852 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:53.961880 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:53.964814 (+ 2934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:53.969822 (+ 5008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:53.969835 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:53.969854 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:53.970424 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:53.971048 (+ 624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:53.971055 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:53.971060 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:53.971395 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:53.972524 (+ 1129us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:53.972534 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:53.972942 (+ 408us) client_negotiation.cc:769] Sending connection context
0806 19:38:53.973266 (+ 324us) client_negotiation.cc:240] Negotiation successful
0806 19:38:53.974029 (+ 763us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"thread_start_us":498,"threads_started":1}
I20260806 19:38:53.978964 25508 heartbeater.cc:347] Connected to a master server at 127.10.25.62:36513
I20260806 19:38:53.979413 25508 heartbeater.cc:464] Registering TS with master...
I20260806 19:38:53.980629 25508 heartbeater.cc:511] Master 127.10.25.62:36513 requested a full tablet report, sending...
I20260806 19:38:53.983579 25314 ts_manager.cc:194] Registered new tserver with Master: 17dc696ed80d4e52beb51b7a6fa9cdac (127.10.25.1:34819)
I20260806 19:38:53.985237 25314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:35563
W20260806 19:38:54.361974 25512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:54.362356 25512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:54.362522 25512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:54.370658 25512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:54.370810 25512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:54.371004 25512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:38:54.382102 25512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:45813
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=41079
--tserver_master_addrs=127.10.25.62:36513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:54.383589 25512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:54.385035 25512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:54.405118 25521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:54.405061 25519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:54.405092 25518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:54.406635 25512 server_base.cc:1034] running on GCE node
I20260806 19:38:54.407470 25512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:54.408648 25512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:54.409881 25512 hybrid_clock.cc:648] HybridClock initialized: now 1786045134409811 us; error 59 us; skew 500 ppm
I20260806 19:38:54.410254 25512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:54 dist-test-slave-ss4n krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045134, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:54.414343 25512 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:38:54.416740 25512 webserver.cc:460] Webserver started at http://127.10.25.2:41079/ using document root <none> and password file <none>
I20260806 19:38:54.417685 25512 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:54.417867 25512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:54.424990 25512 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 19:38:54.430106 25528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:54.432296 25512 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260806 19:38:54.432569 25512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "90058d5c0bcb4f42a4b17d02706b5a6e"
format_stamp: "Formatted at 2026-08-06 19:38:51 on dist-test-slave-ss4n"
server_key: "1b57ea1441450e48bea23fdc9fccdd74"
server_key_iv: "412479fe73d4bac4ea075c0c59165739"
server_key_version: "encryptionkey@0"
I20260806 19:38:54.433846 25512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:54.452445 25512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:54.458201 25512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:54.458528 25512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:54.459930 25512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:38:54.462646 25512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:38:54.462817 25512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:54.462930 25512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:38:54.463003 25512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:54.508275 25512 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:45813
I20260806 19:38:54.508338 25646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:45813 every 8 connection(s)
I20260806 19:38:54.509999 25512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:38:54.520291 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 25512
I20260806 19:38:54.521173 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:37663
--local_ip_for_outbound_sockets=127.10.25.3
--tserver_master_addrs=127.10.25.62:36513
--webserver_port=42037
--webserver_interface=127.10.25.3
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:38:54 dist-test-slave-ss4n krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045134, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:38:54.540122 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:54.514279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:55823 (local address 127.10.25.62:36513)
0806 19:38:54.514932 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:54.514937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:54.516431 (+ 1494us) server_negotiation.cc:407] Connection header received
0806 19:38:54.518798 (+ 2367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:54.518805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:54.518922 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:54.519070 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:54.521766 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:54.522561 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:54.525852 (+ 3291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:54.526333 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:54.526469 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:54.531761 (+ 5292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:54.531768 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:54.531771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:54.531832 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:54.535685 (+ 3853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:54.536762 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:54.536767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:54.536770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:54.536935 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:54.537541 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:54.537546 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:54.537548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:54.537935 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:54.538121 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:54.539056 (+ 935us) server_negotiation.cc:299] Negotiation successful
0806 19:38:54.539343 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":371,"threads_started":1}
I20260806 19:38:54.541771 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:54.515021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:55823)
0806 19:38:54.516184 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:54.516240 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:54.518392 (+ 2152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:54.519388 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:54.519399 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:54.520290 (+ 891us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:54.521384 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:54.521423 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:54.522799 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:54.522808 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:54.525604 (+ 2796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:54.525632 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:54.526637 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:54.526648 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:54.527056 (+ 408us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:54.528008 (+ 952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:54.528058 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:54.531405 (+ 3347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:54.535944 (+ 4539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:54.535962 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:54.535978 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:54.536569 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:54.537083 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:54.537089 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:54.537093 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:54.537348 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:54.538119 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:54.538130 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:54.538564 (+ 434us) client_negotiation.cc:769] Sending connection context
0806 19:38:54.538941 (+ 377us) client_negotiation.cc:240] Negotiation successful
0806 19:38:54.539782 (+ 841us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"thread_start_us":607,"threads_started":1}
I20260806 19:38:54.544317 25647 heartbeater.cc:347] Connected to a master server at 127.10.25.62:36513
I20260806 19:38:54.544842 25647 heartbeater.cc:464] Registering TS with master...
I20260806 19:38:54.546134 25647 heartbeater.cc:511] Master 127.10.25.62:36513 requested a full tablet report, sending...
I20260806 19:38:54.548808 25314 ts_manager.cc:194] Registered new tserver with Master: 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813)
I20260806 19:38:54.550437 25314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:55823
W20260806 19:38:54.914211 25651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:54.914645 25651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:54.914793 25651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:54.923897 25651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:54.924057 25651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:54.924189 25651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:38:54.936338 25651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:37663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=42037
--tserver_master_addrs=127.10.25.62:36513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:54.938056 25651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:54.939908 25651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:54.958355 25656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:54.958310 25657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:54.958268 25659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:54.960031 25651 server_base.cc:1034] running on GCE node
I20260806 19:38:54.960858 25651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:54.962179 25651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:54.963486 25651 hybrid_clock.cc:648] HybridClock initialized: now 1786045134963415 us; error 64 us; skew 500 ppm
I20260806 19:38:54.963930 25651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:54 dist-test-slave-ss4n krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045134, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:54.968786 25651 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:38:54.970836 25651 webserver.cc:460] Webserver started at http://127.10.25.3:42037/ using document root <none> and password file <none>
I20260806 19:38:54.971585 25651 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:54.971783 25651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:54.978969 25651 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 19:38:54.982340 25667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:54.983990 25651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 19:38:54.984205 25651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "b8da86666b7d4778807c51842118d38c"
format_stamp: "Formatted at 2026-08-06 19:38:51 on dist-test-slave-ss4n"
server_key: "5aa9ac2e3b535f9d54aae9d6c00584f2"
server_key_iv: "9509ed937a51c329027c0afdac872da6"
server_key_version: "encryptionkey@0"
I20260806 19:38:54.985280 25651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:54.989836 25508 heartbeater.cc:503] Master 127.10.25.62:36513 was elected leader, sending a full tablet report...
I20260806 19:38:55.019016 25651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:55.025477 25651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:55.025870 25651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:55.027149 25651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:38:55.029084 25651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:38:55.029191 25651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:55.029273 25651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:38:55.029328 25651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:55.075944 25651 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:37663
I20260806 19:38:55.076159 25783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:37663 every 8 connection(s)
I20260806 19:38:55.077796 25651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:38:55.079356 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 25651
Aug 06 19:38:55 dist-test-slave-ss4n krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045134, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:38:55.101933 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.081247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36298 (local address 127.10.25.62:36513)
0806 19:38:55.082372 (+ 1125us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:55.082377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:55.082395 (+ 18us) server_negotiation.cc:407] Connection header received
0806 19:38:55.082463 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:55.082470 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:55.082553 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:55.082684 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:55.084818 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.086012 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.087618 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.088378 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.088528 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.091775 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:55.091783 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:55.091793 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:55.091830 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:55.095440 (+ 3610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:55.096832 (+ 1392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:55.096842 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:55.096849 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:55.097019 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:55.097737 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:55.097745 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:55.097751 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:55.098418 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:55.099085 (+ 667us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:55.100979 (+ 1894us) server_negotiation.cc:299] Negotiation successful
0806 19:38:55.101360 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":975,"spinlock_wait_cycles":1408,"thread_start_us":4017,"threads_started":1}
I20260806 19:38:55.110121 25787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.081786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:54983 (local address 127.10.25.62:36513)
0806 19:38:55.085627 (+ 3841us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:55.085632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:55.085651 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:38:55.087905 (+ 2254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:55.087910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:55.088013 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:55.088142 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:55.090544 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.091521 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.094028 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.094898 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.095117 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.100802 (+ 5685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:55.100808 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:55.100812 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:55.100903 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:55.105183 (+ 4280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:55.106516 (+ 1333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:55.106523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:55.106528 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:55.106687 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:55.107250 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:55.107256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:55.107260 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:55.108061 (+ 801us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:55.108278 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:55.109132 (+ 854us) server_negotiation.cc:299] Negotiation successful
0806 19:38:55.109464 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3654,"thread_start_us":1968,"threads_started":1}
I20260806 19:38:55.110945 25789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.083274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:54983)
0806 19:38:55.084459 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:55.084535 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:55.087471 (+ 2936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:55.088530 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:55.088545 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:55.089390 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:55.090306 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:55.090338 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.091708 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.091715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:55.093696 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:55.093731 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.095287 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.095300 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:55.095743 (+ 443us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.096684 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:55.096727 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:55.100442 (+ 3715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:55.105381 (+ 4939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:55.105396 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:55.105422 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:55.106313 (+ 891us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:55.106836 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:55.106843 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:55.106847 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:55.107099 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:55.108244 (+ 1145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:55.108254 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:55.108674 (+ 420us) client_negotiation.cc:769] Sending connection context
0806 19:38:55.108943 (+ 269us) client_negotiation.cc:240] Negotiation successful
0806 19:38:55.109429 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"thread_start_us":459,"threads_started":1}
I20260806 19:38:55.112896 25784 heartbeater.cc:347] Connected to a master server at 127.10.25.62:36513
I20260806 19:38:55.113334 25784 heartbeater.cc:464] Registering TS with master...
I20260806 19:38:55.114387 25784 heartbeater.cc:511] Master 127.10.25.62:36513 requested a full tablet report, sending...
I20260806 19:38:55.116459 25314 ts_manager.cc:194] Registered new tserver with Master: b8da86666b7d4778807c51842118d38c (127.10.25.3:37663)
I20260806 19:38:55.118040 25314 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:54983
I20260806 19:38:55.122339 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:38:55.131770 25787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.123893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36314 (local address 127.10.25.62:36513)
0806 19:38:55.124291 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:55.124301 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:55.124741 (+ 440us) server_negotiation.cc:407] Connection header received
0806 19:38:55.125052 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:55.125057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:55.125243 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:55.125445 (+ 202us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:38:55.126044 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.126961 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.128879 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.129433 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.129690 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.130362 (+ 672us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:38:55.130997 (+ 635us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:38:55.131183 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:55.131314 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 19:38:55.131428 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 19:38:55.133069 25314 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:38:55.141819 25314 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:38:55.171835 25797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.157829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:34819 (local address 127.0.0.1:41498)
0806 19:38:55.159806 (+ 1977us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:55.159877 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:55.160225 (+ 348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:55.163345 (+ 3120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:55.163353 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:55.163392 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:55.164043 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:55.164065 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.166137 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.166149 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:55.168387 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:55.168401 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.170557 (+ 2156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.170565 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:55.170811 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.170879 (+ 68us) client_negotiation.cc:769] Sending connection context
0806 19:38:55.171075 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 19:38:55.171327 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1589,"mutex_wait_us":50,"thread_start_us":3447,"threads_started":1}
I20260806 19:38:55.172679 25796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.158145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41498 (local address 127.10.25.1:34819)
0806 19:38:55.161571 (+ 3426us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:55.161589 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:55.162233 (+ 644us) server_negotiation.cc:407] Connection header received
0806 19:38:55.162375 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:55.162388 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:55.163062 (+ 674us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:55.163303 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:55.164295 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.165993 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.168575 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.170291 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.170435 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.170718 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:55.171605 (+ 887us) server_negotiation.cc:299] Negotiation successful
0806 19:38:55.171817 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2731,"thread_start_us":2525,"threads_started":1}
I20260806 19:38:55.173319 25798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.159132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48110 (local address 127.10.25.2:45813)
0806 19:38:55.162959 (+ 3827us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:55.162968 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:55.162998 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:38:55.163102 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:55.163113 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:55.163353 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:55.163581 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:55.164621 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.166820 (+ 2199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.169628 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.172068 (+ 2440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.172310 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.172512 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:55.172769 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 19:38:55.172960 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3659,"thread_start_us":3492,"threads_started":1}
I20260806 19:38:55.174041 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.158361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:45813 (local address 127.0.0.1:48110)
0806 19:38:55.161887 (+ 3526us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:55.161911 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:55.162040 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:55.163683 (+ 1643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:55.163689 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:55.163724 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:55.164377 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:55.164392 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.167000 (+ 2608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.167007 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:55.169359 (+ 2352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:55.169374 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.172343 (+ 2969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.172351 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:55.172547 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.172589 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 19:38:55.172712 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 19:38:55.172878 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3384,"mutex_wait_us":1,"thread_start_us":985,"threads_started":1}
I20260806 19:38:55.175483 25800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.163649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:37663 (local address 127.0.0.1:34998)
0806 19:38:55.164915 (+ 1266us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:55.164937 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:55.165078 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:55.165757 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:55.165764 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:55.165792 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:55.166310 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:55.166325 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.168857 (+ 2532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.168862 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:55.171008 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:55.171027 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.173632 (+ 2605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.173638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:55.173764 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.173827 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 19:38:55.173925 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 19:38:55.174095 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1127,"thread_start_us":959,"threads_started":1}
I20260806 19:38:55.176345 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.163315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34998 (local address 127.10.25.3:37663)
0806 19:38:55.164419 (+ 1104us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:55.164429 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:55.165054 (+ 625us) server_negotiation.cc:407] Connection header received
0806 19:38:55.165247 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:55.165255 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:55.165471 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:55.165689 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:55.166752 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.168682 (+ 1930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.171222 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.173014 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.173150 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.173367 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:55.175905 (+ 2538us) server_negotiation.cc:299] Negotiation successful
0806 19:38:55.176071 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":897,"thread_start_us":656,"threads_started":1}
I20260806 19:38:55.177999 25579 tablet_service.cc:1467] Processing CreateTablet for tablet 1c3cb447dae54220a81eeb25a57d485c (DEFAULT_TABLE table=test-table [id=863a501db0e04398b574ea9ce401b934]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:38:55.178112 25441 tablet_service.cc:1467] Processing CreateTablet for tablet 1c3cb447dae54220a81eeb25a57d485c (DEFAULT_TABLE table=test-table [id=863a501db0e04398b574ea9ce401b934]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:38:55.179709 25441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c3cb447dae54220a81eeb25a57d485c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:38:55.179697 25579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c3cb447dae54220a81eeb25a57d485c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:38:55.180181 25716 tablet_service.cc:1467] Processing CreateTablet for tablet 1c3cb447dae54220a81eeb25a57d485c (DEFAULT_TABLE table=test-table [id=863a501db0e04398b574ea9ce401b934]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:38:55.181458 25716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c3cb447dae54220a81eeb25a57d485c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:38:55.198560 25801 tablet_bootstrap.cc:492] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c: Bootstrap starting.
I20260806 19:38:55.204049 25801 tablet_bootstrap.cc:654] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c: Neither blocks nor log segments found. Creating new log.
I20260806 19:38:55.205343 25801 log.cc:824] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:55.206658 25802 tablet_bootstrap.cc:492] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac: Bootstrap starting.
I20260806 19:38:55.207800 25803 tablet_bootstrap.cc:492] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e: Bootstrap starting.
I20260806 19:38:55.211313 25801 tablet_bootstrap.cc:492] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c: No bootstrap required, opened a new log
I20260806 19:38:55.211876 25801 ts_tablet_manager.cc:1397] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260806 19:38:55.212046 25802 tablet_bootstrap.cc:654] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac: Neither blocks nor log segments found. Creating new log.
I20260806 19:38:55.212670 25803 tablet_bootstrap.cc:654] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e: Neither blocks nor log segments found. Creating new log.
I20260806 19:38:55.213804 25802 log.cc:824] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:55.214109 25803 log.cc:824] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:55.217758 25803 tablet_bootstrap.cc:492] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e: No bootstrap required, opened a new log
I20260806 19:38:55.218021 25802 tablet_bootstrap.cc:492] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac: No bootstrap required, opened a new log
I20260806 19:38:55.218158 25803 ts_tablet_manager.cc:1397] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260806 19:38:55.218336 25802 ts_tablet_manager.cc:1397] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260806 19:38:55.224888 25801 raft_consensus.cc:348] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:55.225564 25801 raft_consensus.cc:374] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:38:55.225692 25801 raft_consensus.cc:729] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8da86666b7d4778807c51842118d38c, State: Initialized, Role: FOLLOWER
I20260806 19:38:55.226528 25801 consensus_queue.cc:260] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:55.227108 25803 raft_consensus.cc:348] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:55.227756 25803 raft_consensus.cc:374] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:38:55.227921 25803 raft_consensus.cc:729] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90058d5c0bcb4f42a4b17d02706b5a6e, State: Initialized, Role: FOLLOWER
I20260806 19:38:55.228578 25784 heartbeater.cc:503] Master 127.10.25.62:36513 was elected leader, sending a full tablet report...
I20260806 19:38:55.228953 25801 ts_tablet_manager.cc:1428] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c: Time spent starting tablet: real 0.017s user 0.014s sys 0.004s
I20260806 19:38:55.228880 25803 consensus_queue.cc:260] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:55.230268 25647 heartbeater.cc:503] Master 127.10.25.62:36513 was elected leader, sending a full tablet report...
I20260806 19:38:55.231722 25803 ts_tablet_manager.cc:1428] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260806 19:38:55.232699 25802 raft_consensus.cc:348] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:55.233301 25802 raft_consensus.cc:374] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:38:55.233436 25802 raft_consensus.cc:729] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17dc696ed80d4e52beb51b7a6fa9cdac, State: Initialized, Role: FOLLOWER
I20260806 19:38:55.234176 25802 consensus_queue.cc:260] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:55.236480 25802 ts_tablet_manager.cc:1428] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac: Time spent starting tablet: real 0.018s user 0.016s sys 0.000s
W20260806 19:38:55.263025 25648 tablet.cc:2369] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:38:55.329883 25785 tablet.cc:2369] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:38:55.402487 25810 raft_consensus.cc:482] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:38:55.402837 25810 raft_consensus.cc:504] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:55.405465 25810 leader_election.cc:288] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813), b8da86666b7d4778807c51842118d38c (127.10.25.3:37663)
I20260806 19:38:55.414774 25798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.406079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:47631 (local address 127.10.25.2:45813)
0806 19:38:55.406369 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:55.406378 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:55.406727 (+ 349us) server_negotiation.cc:407] Connection header received
0806 19:38:55.406971 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:55.406977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:55.407053 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:55.407165 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:55.408058 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.409192 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.411673 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.413176 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.413323 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.413428 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:55.414349 (+ 921us) server_negotiation.cc:299] Negotiation successful
0806 19:38:55.414526 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 19:38:55.414857 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.405863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:45813 (local address 127.10.25.1:47631)
0806 19:38:55.406607 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:55.406624 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:55.406814 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:55.407388 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:55.407392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:55.407425 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:55.407831 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:55.407840 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.409345 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.409351 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:55.411437 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:55.411452 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.413625 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.413635 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:55.414026 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.414087 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 19:38:55.414235 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 19:38:55.414417 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":367,"threads_started":1}
I20260806 19:38:55.416357 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.407053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:55603 (local address 127.10.25.3:37663)
0806 19:38:55.407294 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:55.407303 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:55.407904 (+ 601us) server_negotiation.cc:407] Connection header received
0806 19:38:55.408098 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:55.408102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:55.408283 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:55.408383 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:38:55.409070 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.410395 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.413344 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.415198 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.415402 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.415540 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:55.415917 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 19:38:55.416072 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 19:38:55.416518 25812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.406799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:37663 (local address 127.10.25.1:55603)
0806 19:38:55.407771 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:55.407791 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:55.407920 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:55.408446 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:55.408451 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:55.408481 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:38:55.408872 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:55.408884 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.410659 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.410664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:55.412908 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:55.412948 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.415484 (+ 2536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.415491 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:55.415727 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.415756 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 19:38:55.415883 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 19:38:55.416056 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"mutex_wait_us":1,"thread_start_us":573,"threads_started":1}
I20260806 19:38:55.416257 25600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3cb447dae54220a81eeb25a57d485c" candidate_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" is_pre_election: true
I20260806 19:38:55.416996 25600 raft_consensus.cc:2454] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17dc696ed80d4e52beb51b7a6fa9cdac in term 0.
I20260806 19:38:55.417795 25736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3cb447dae54220a81eeb25a57d485c" candidate_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8da86666b7d4778807c51842118d38c" is_pre_election: true
I20260806 19:38:55.418004 25394 leader_election.cc:302] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17dc696ed80d4e52beb51b7a6fa9cdac, 90058d5c0bcb4f42a4b17d02706b5a6e; no voters:
I20260806 19:38:55.418533 25810 raft_consensus.cc:2781] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:38:55.418517 25736 raft_consensus.cc:2454] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17dc696ed80d4e52beb51b7a6fa9cdac in term 0.
I20260806 19:38:55.418682 25810 raft_consensus.cc:482] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:38:55.418814 25810 raft_consensus.cc:3037] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:38:55.422255 25810 raft_consensus.cc:504] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:55.423094 25810 leader_election.cc:288] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 1 election: Requested vote from peers 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813), b8da86666b7d4778807c51842118d38c (127.10.25.3:37663)
I20260806 19:38:55.423717 25600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3cb447dae54220a81eeb25a57d485c" candidate_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e"
I20260806 19:38:55.423736 25736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3cb447dae54220a81eeb25a57d485c" candidate_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8da86666b7d4778807c51842118d38c"
I20260806 19:38:55.423972 25600 raft_consensus.cc:3037] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:38:55.423964 25736 raft_consensus.cc:3037] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:38:55.427225 25600 raft_consensus.cc:2454] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17dc696ed80d4e52beb51b7a6fa9cdac in term 1.
I20260806 19:38:55.427561 25736 raft_consensus.cc:2454] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17dc696ed80d4e52beb51b7a6fa9cdac in term 1.
I20260806 19:38:55.427892 25394 leader_election.cc:302] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17dc696ed80d4e52beb51b7a6fa9cdac, 90058d5c0bcb4f42a4b17d02706b5a6e; no voters:
I20260806 19:38:55.428339 25810 raft_consensus.cc:2781] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:38:55.428983 25810 raft_consensus.cc:686] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 LEADER]: Becoming Leader. State: Replica: 17dc696ed80d4e52beb51b7a6fa9cdac, State: Running, Role: LEADER
I20260806 19:38:55.429606 25810 consensus_queue.cc:237] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:55.434813 25314 catalog_manager.cc:5495] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac reported cstate change: term changed from 0 to 1, leader changed from <none> to 17dc696ed80d4e52beb51b7a6fa9cdac (127.10.25.1). New cstate: current_term: 1 leader_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } health_report { overall_health: UNKNOWN } } }
W20260806 19:38:55.445597 25509 tablet.cc:2369] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:38:55.479813 25796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.472857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41508 (local address 127.10.25.1:34819)
0806 19:38:55.473088 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:55.473093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:55.473115 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:38:55.473226 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:55.473230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:55.473321 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:55.473470 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:38:55.474081 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.474874 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.476934 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.477594 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.477720 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.478432 (+ 712us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:38:55.479066 (+ 634us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:38:55.479188 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:55.479440 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 19:38:55.479566 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 19:38:55.490453 25736 raft_consensus.cc:1261] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c [term 1 FOLLOWER]: Refusing update from remote peer 17dc696ed80d4e52beb51b7a6fa9cdac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:38:55.490438 25600 raft_consensus.cc:1261] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e [term 1 FOLLOWER]: Refusing update from remote peer 17dc696ed80d4e52beb51b7a6fa9cdac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:38:55.491828 25813 consensus_queue.cc:1035] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:38:55.492448 25810 consensus_queue.cc:1035] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:38:55.512694 25819 mvcc.cc:205] Tried to move back new op lower bound from 7315640874953940992 to 7315640874723995648. Current Snapshot: MvccSnapshot[applied={T|T < 7315640874953940992 or (T in {7315640874953940992})}]
I20260806 19:38:55.525355 25314 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:38:55.525748 25314 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:38:55.531301 25314 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9: Sending DeleteTablet for 3 replicas of tablet 1c3cb447dae54220a81eeb25a57d485c
I20260806 19:38:55.533556 25441 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c3cb447dae54220a81eeb25a57d485c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:38:55 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:41498
I20260806 19:38:55.533524 25579 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c3cb447dae54220a81eeb25a57d485c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:38:55 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:48110
I20260806 19:38:55.534596 25716 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c3cb447dae54220a81eeb25a57d485c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:38:55 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:34998
I20260806 19:38:55.540735 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 25352
I20260806 19:38:55.558935 25828 tablet_replica.cc:330] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e: stopping tablet replica
I20260806 19:38:55.559494 25828 raft_consensus.cc:2229] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:38:55.560117 25828 raft_consensus.cc:2258] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:38:55.560988 25829 tablet_replica.cc:330] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c: stopping tablet replica
I20260806 19:38:55.561582 25829 raft_consensus.cc:2229] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:38:55.562150 25829 raft_consensus.cc:2258] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:38:55.563123 25827 tablet_replica.cc:330] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac: stopping tablet replica
I20260806 19:38:55.563572 25827 raft_consensus.cc:2229] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:38:55.564306 25827 raft_consensus.cc:2258] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:38:55.566089 25829 ts_tablet_manager.cc:1905] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:38:55.566838 25828 ts_tablet_manager.cc:1905] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:38:55.568768 25796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.541764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41518 (local address 127.10.25.1:34819)
0806 19:38:55.541944 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:55.541952 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:55.541988 (+ 36us) server_negotiation.cc:407] Connection header received
0806 19:38:55.542336 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:55.542345 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:55.542614 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:55.542889 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:55.545534 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.547280 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.549284 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.550409 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.550580 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.558339 (+ 7759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:55.558364 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:55.558392 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:55.558464 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:55.563773 (+ 5309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:55.564991 (+ 1218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:55.565003 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:55.565006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:55.565262 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:55.565836 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:55.565844 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:55.565848 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:55.566344 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:55.566660 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:55.568078 (+ 1418us) server_negotiation.cc:299] Negotiation successful
0806 19:38:55.568372 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 19:38:55.569592 25503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:55.575693 25828 ts_tablet_manager.cc:1918] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:38:55.575915 25828 log.cc:1196] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/1c3cb447dae54220a81eeb25a57d485c
I20260806 19:38:55.576465 25828 ts_tablet_manager.cc:1939] T 1c3cb447dae54220a81eeb25a57d485c P 90058d5c0bcb4f42a4b17d02706b5a6e: Deleting consensus metadata
I20260806 19:38:55.576900 25829 ts_tablet_manager.cc:1918] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:38:55.577090 25829 log.cc:1196] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/1c3cb447dae54220a81eeb25a57d485c
I20260806 19:38:55.577747 25829 ts_tablet_manager.cc:1939] T 1c3cb447dae54220a81eeb25a57d485c P b8da86666b7d4778807c51842118d38c: Deleting consensus metadata
I20260806 19:38:55.579238 25300 catalog_manager.cc:4841] TS 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813): tablet 1c3cb447dae54220a81eeb25a57d485c (table test-table [id=863a501db0e04398b574ea9ce401b934]) successfully deleted
I20260806 19:38:55.579667 25300 catalog_manager.cc:4841] TS b8da86666b7d4778807c51842118d38c (127.10.25.3:37663): tablet 1c3cb447dae54220a81eeb25a57d485c (table test-table [id=863a501db0e04398b574ea9ce401b934]) successfully deleted
I20260806 19:38:55.678654 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 25352
W20260806 19:38:55.703945 25298 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.25.1:34819 (error 108)
I20260806 19:38:55.704218 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 25512
I20260806 19:38:55.706852 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.705883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:34819 (local address 127.0.0.1:41530)
0806 19:38:55.706445 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:55.706597 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":284,"threads_started":1}
W20260806 19:38:55.707161 25298 catalog_manager.cc:4584] TS 17dc696ed80d4e52beb51b7a6fa9cdac (127.10.25.1:34819): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1c3cb447dae54220a81eeb25a57d485c: Network error: Client connection negotiation failed: client connection to 127.10.25.1:34819: connect: Connection refused (error 111)
I20260806 19:38:55.721153 25798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.705432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48118 (local address 127.10.25.2:45813)
0806 19:38:55.705706 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:55.705713 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:55.705797 (+ 84us) server_negotiation.cc:407] Connection header received
0806 19:38:55.705960 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:55.705966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:55.706102 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:55.706249 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:55.708010 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.708876 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.710393 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.710875 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.710998 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.714224 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:55.714244 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:55.714267 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:55.714319 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:55.717844 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:55.718790 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:55.718798 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:55.718801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:55.718947 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:55.719427 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:55.719432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:55.719434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:55.719852 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:55.720190 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:55.720511 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 19:38:55.720768 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260806 19:38:55.722126 25642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:55.761130 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.760509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:34819 (local address 127.0.0.1:41540)
0806 19:38:55.760790 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:55.760914 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260806 19:38:55.762517 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.762010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:34819 (local address 127.0.0.1:41544)
0806 19:38:55.762264 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:55.762345 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260806 19:38:55.822762 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 25512
I20260806 19:38:55.823681 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.822854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:34819 (local address 127.0.0.1:41560)
0806 19:38:55.823186 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:55.823476 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200}
I20260806 19:38:55.825922 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.825310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:34819 (local address 127.0.0.1:41564)
0806 19:38:55.825600 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:55.825710 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 19:38:55.846650 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 25651
I20260806 19:38:55.863893 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.847602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35002 (local address 127.10.25.3:37663)
0806 19:38:55.847944 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:55.847953 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:55.848111 (+ 158us) server_negotiation.cc:407] Connection header received
0806 19:38:55.848523 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:55.848529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:55.848643 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:55.848779 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:55.850806 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.851635 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.853315 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:55.853864 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:55.854035 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:55.857217 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:55.857236 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:55.857254 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:55.857299 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:55.860555 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:55.861414 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:55.861432 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:55.861435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:55.861688 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:55.862159 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:55.862174 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:55.862178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:55.862572 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:55.862888 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:55.863231 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 19:38:55.863497 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260806 19:38:55.864722 25779 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:55.907157 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.906523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:34819 (local address 127.0.0.1:41568)
0806 19:38:55.906883 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:55.906982 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217}
I20260806 19:38:55.908447 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:55.908009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:34819 (local address 127.0.0.1:41572)
0806 19:38:55.908217 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:55.908291 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260806 19:38:55.963340 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 25651
I20260806 19:38:55.985669 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 25279
I20260806 19:38:55.986881 25345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:38:56.111989 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:56.110818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:34819 (local address 127.0.0.1:41582)
0806 19:38:56.111605 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:56.111747 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:34819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372}
I20260806 19:38:56.112291 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 25279
I20260806 19:38:56.137974 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:36513
--webserver_interface=127.10.25.62
--webserver_port=41781
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:36513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:38:56.514933 25836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:56.515298 25836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:56.515399 25836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:56.523159 25836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:38:56.523303 25836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:56.523353 25836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:56.523406 25836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:38:56.523444 25836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:38:56.534679 25836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:36513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:36513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=41781
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:56.536087 25836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:56.537385 25836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:56.550987 25842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:56.551020 25844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:56.551085 25841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:56.552173 25836 server_base.cc:1034] running on GCE node
I20260806 19:38:56.553162 25836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:56.555183 25836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:56.556531 25836 hybrid_clock.cc:648] HybridClock initialized: now 1786045136556486 us; error 38 us; skew 500 ppm
I20260806 19:38:56.556962 25836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:56 dist-test-slave-ss4n krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045136, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:56.560810 25836 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:38:56.563570 25836 webserver.cc:460] Webserver started at http://127.10.25.62:41781/ using document root <none> and password file <none>
I20260806 19:38:56.564451 25836 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:56.564625 25836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:56.570990 25836 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 19:38:56.577188 25852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:56.578909 25836 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 19:38:56.579161 25836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "30875fa4eb10456d9b4873efd27524a9"
format_stamp: "Formatted at 2026-08-06 19:38:49 on dist-test-slave-ss4n"
server_key: "077204bc75284e6ad342f177201aba24"
server_key_iv: "d7eb87a401d837b0159bd6573ccc4faa"
server_key_version: "encryptionkey@0"
I20260806 19:38:56.580394 25836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:56.594352 25836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:56.599370 25836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:56.599728 25836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:56.622468 25836 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:36513
I20260806 19:38:56.622471 25906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:36513 every 8 connection(s)
I20260806 19:38:56.624163 25836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:38:56.631628 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 25836
I20260806 19:38:56.632169 25907 sys_catalog.cc:263] Verifying existing consensus state
I20260806 19:38:56.632712 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:34819
--local_ip_for_outbound_sockets=127.10.25.1
--tserver_master_addrs=127.10.25.62:36513
--webserver_port=45283
--webserver_interface=127.10.25.1
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:38:56.638752 25907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9: Bootstrap starting.
I20260806 19:38:56.663954 25907 log.cc:824] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:56.687073 25907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:38:56.687773 25907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9: Bootstrap complete.
I20260806 19:38:56.698508 25907 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } }
I20260806 19:38:56.699576 25907 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30875fa4eb10456d9b4873efd27524a9, State: Initialized, Role: FOLLOWER
I20260806 19:38:56.700452 25907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } }
I20260806 19:38:56.700748 25907 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:38:56.700891 25907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:38:56.701122 25907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 2 FOLLOWER]: Advancing to term 3
I20260806 19:38:56.705663 25907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } }
I20260806 19:38:56.706305 25907 leader_election.cc:302] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30875fa4eb10456d9b4873efd27524a9; no voters:
I20260806 19:38:56.707062 25907 leader_election.cc:288] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 19:38:56.707221 25911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 19:38:56.708676 25911 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [term 3 LEADER]: Becoming Leader. State: Replica: 30875fa4eb10456d9b4873efd27524a9, State: Running, Role: LEADER
I20260806 19:38:56.709326 25911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } }
I20260806 19:38:56.709728 25907 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:38:56.713527 25913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30875fa4eb10456d9b4873efd27524a9. Latest consensus state: current_term: 3 leader_uuid: "30875fa4eb10456d9b4873efd27524a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } } }
I20260806 19:38:56.714402 25913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [sys.catalog]: This master's current role is: LEADER
I20260806 19:38:56.715078 25912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "30875fa4eb10456d9b4873efd27524a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30875fa4eb10456d9b4873efd27524a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36513 } } }
I20260806 19:38:56.716377 25912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30875fa4eb10456d9b4873efd27524a9 [sys.catalog]: This master's current role is: LEADER
I20260806 19:38:56.728799 25917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:38:56.740983 25917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:38:56.743636 25917 catalog_manager.cc:1259] Loaded cluster ID: 9a9b65f399994c3fbf4522d93bbbe58a
I20260806 19:38:56.743770 25917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 19:38:56.746618 25917 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fdeaf1ca9ad google::LogMessage::Fail() at ??:0
@ 0x7fdeaf1cd88a google::LogMessage::SendToLog() at ??:0
@ 0x7fdeaf1ca4de google::LogMessage::Flush() at ??:0
@ 0x7fdeaf1cba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fdec4bfd512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fdec4bfa976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fdec4c82610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fdec4c82439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fdeb1e16345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fdeb1e1991c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fdeb1e19799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fdeb1df4eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdeb50ce6db start_thread at ??:0
@ 0x7fdeae06461f clone at ??:0
W20260806 19:38:57.063766 25909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:57.064153 25909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:57.064337 25909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:57.074048 25909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:57.074267 25909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:57.074477 25909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:38:57.086588 25909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:34819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=45283
--tserver_master_addrs=127.10.25.62:36513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:57.088203 25909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:57.089852 25909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:57.104758 25953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:57.104696 25951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:57.105201 25950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:57.105576 25909 server_base.cc:1034] running on GCE node
I20260806 19:38:57.106292 25909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:57.107399 25909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:57.108664 25909 hybrid_clock.cc:648] HybridClock initialized: now 1786045137108617 us; error 62 us; skew 500 ppm
I20260806 19:38:57.109015 25909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:57 dist-test-slave-ss4n krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045137, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:57.112614 25909 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:38:57.114380 25909 webserver.cc:460] Webserver started at http://127.10.25.1:45283/ using document root <none> and password file <none>
I20260806 19:38:57.115162 25909 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:57.115325 25909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:57.121328 25909 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 19:38:57.125305 25961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:57.127218 25909 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260806 19:38:57.127488 25909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "17dc696ed80d4e52beb51b7a6fa9cdac"
format_stamp: "Formatted at 2026-08-06 19:38:50 on dist-test-slave-ss4n"
server_key: "ef68a4bf0f4386e7d7c905718bc25e48"
server_key_iv: "f2ed245b745eb7109ce8199be75f4321"
server_key_version: "encryptionkey@0"
I20260806 19:38:57.128769 25909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:57.150028 25909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:57.155553 25909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:57.155964 25909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:57.157250 25909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:38:57.159606 25968 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260806 19:38:57.166676 25909 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 19:38:57.166884 25909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.000s
I20260806 19:38:57.167061 25909 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 19:38:57.170625 25909 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 19:38:57.170763 25909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.003s sys 0.000s
I20260806 19:38:57.171340 25968 tablet_bootstrap.cc:492] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac: Bootstrap starting.
I20260806 19:38:57.181205 25968 log.cc:824] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac: Log is configured to *not* fsync() on all Append() calls
I20260806 19:38:57.198695 25968 tablet_bootstrap.cc:492] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:38:57.199374 25968 tablet_bootstrap.cc:492] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac: Bootstrap complete.
I20260806 19:38:57.199880 25968 ts_tablet_manager.cc:1397] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac: Time spent bootstrapping tablet: real 0.029s user 0.022s sys 0.004s
I20260806 19:38:57.211494 25968 raft_consensus.cc:348] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:57.213534 25968 raft_consensus.cc:729] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17dc696ed80d4e52beb51b7a6fa9cdac, State: Initialized, Role: FOLLOWER
I20260806 19:38:57.214434 25968 consensus_queue.cc:260] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:57.216507 25968 ts_tablet_manager.cc:1428] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac: Time spent starting tablet: real 0.016s user 0.014s sys 0.004s
I20260806 19:38:57.223428 25909 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:34819
I20260806 19:38:57.223482 26080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:34819 every 8 connection(s)
I20260806 19:38:57.224898 25909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:38:57.229411 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 25909
I20260806 19:38:57.230271 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:45813
--local_ip_for_outbound_sockets=127.10.25.2
--tserver_master_addrs=127.10.25.62:36513
--webserver_port=41079
--webserver_interface=127.10.25.2
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:38:57.230688 26083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:57.228251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:33227)
0806 19:38:57.229250 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:57.229596 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":493,"threads_started":1}
W20260806 19:38:57.231485 26081 heartbeater.cc:644] Failed to heartbeat to 127.10.25.62:36513 (0 consecutive failures): Network error: Failed to ping master at 127.10.25.62:36513: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
I20260806 19:38:57.232628 26083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:57.232230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:47135)
0806 19:38:57.232427 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:57.232501 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 19:38:57.233639 26083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:57.233178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:44841)
0806 19:38:57.233355 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:57.233430 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 19:38:57.233884 26081 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 19:38:57.600237 26084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:57.600566 26084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:57.600800 26084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:57.608451 26084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:57.608603 26084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:57.608780 26084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:38:57.619472 26084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:45813
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=41079
--tserver_master_addrs=127.10.25.62:36513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:57.620985 26084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:57.622479 26084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:57.638576 26090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:57.638624 26091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:57.638674 26093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:57.640223 26084 server_base.cc:1034] running on GCE node
I20260806 19:38:57.641093 26084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:57.642566 26084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:57.643849 26084 hybrid_clock.cc:648] HybridClock initialized: now 1786045137643750 us; error 71 us; skew 500 ppm
I20260806 19:38:57.644289 26084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:57 dist-test-slave-ss4n krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045137, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:57.648227 26084 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:38:57.650267 26084 webserver.cc:460] Webserver started at http://127.10.25.2:41079/ using document root <none> and password file <none>
I20260806 19:38:57.651088 26084 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:57.651264 26084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:57.658282 26084 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 19:38:57.662037 26100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:57.663908 26084 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 19:38:57.664109 26084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "90058d5c0bcb4f42a4b17d02706b5a6e"
format_stamp: "Formatted at 2026-08-06 19:38:51 on dist-test-slave-ss4n"
server_key: "1b57ea1441450e48bea23fdc9fccdd74"
server_key_iv: "412479fe73d4bac4ea075c0c59165739"
server_key_version: "encryptionkey@0"
I20260806 19:38:57.665169 26084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:57.691630 26084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:57.696666 26084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:57.696943 26084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:57.698115 26084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:38:57.699743 26084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:38:57.699849 26084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:57.699929 26084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:38:57.699977 26084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:57.742735 26084 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:45813
I20260806 19:38:57.743022 26215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:45813 every 8 connection(s)
I20260806 19:38:57.744498 26084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:38:57.750166 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:57.747957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:49799)
0806 19:38:57.748824 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:57.749067 (+ 243us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":352,"threads_started":1}
W20260806 19:38:57.751036 26216 heartbeater.cc:644] Failed to heartbeat to 127.10.25.62:36513 (0 consecutive failures): Network error: Failed to ping master at 127.10.25.62:36513: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
I20260806 19:38:57.752025 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:57.751630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:44525)
0806 19:38:57.751815 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:57.751904 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 19:38:57.753723 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:57.752796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:43001)
0806 19:38:57.753186 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:57.753528 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260806 19:38:57.753876 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 26084
W20260806 19:38:57.754110 26216 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 19:38:57.754644 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:37663
--local_ip_for_outbound_sockets=127.10.25.3
--tserver_master_addrs=127.10.25.62:36513
--webserver_port=42037
--webserver_interface=127.10.25.3
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:38:58.144642 26219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:38:58.144979 26219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:38:58.145226 26219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:38:58.153445 26219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:38:58.153625 26219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:38:58.153827 26219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:38:58.164968 26219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:37663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=42037
--tserver_master_addrs=127.10.25.62:36513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:38:58.166502 26219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:38:58.167881 26219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:38:58.184341 26224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:58.184278 26227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:38:58.184376 26225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:38:58.185343 26219 server_base.cc:1034] running on GCE node
I20260806 19:38:58.186141 26219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:38:58.187425 26219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:38:58.188669 26219 hybrid_clock.cc:648] HybridClock initialized: now 1786045138188609 us; error 57 us; skew 500 ppm
I20260806 19:38:58.189081 26219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:38:58 dist-test-slave-ss4n krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045138, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:38:58.193130 26219 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:38:58.195125 26219 webserver.cc:460] Webserver started at http://127.10.25.3:42037/ using document root <none> and password file <none>
I20260806 19:38:58.195941 26219 fs_manager.cc:359] Metadata directory not provided
I20260806 19:38:58.196076 26219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:38:58.203088 26219 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:38:58.206496 26235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:58.208501 26219 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:38:58.208783 26219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "b8da86666b7d4778807c51842118d38c"
format_stamp: "Formatted at 2026-08-06 19:38:51 on dist-test-slave-ss4n"
server_key: "5aa9ac2e3b535f9d54aae9d6c00584f2"
server_key_iv: "9509ed937a51c329027c0afdac872da6"
server_key_version: "encryptionkey@0"
I20260806 19:38:58.210170 26219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:38:58.230469 26219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:38:58.235661 26240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:58.234587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:56843)
0806 19:38:58.235236 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:58.235352 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":305,"threads_started":1}
I20260806 19:38:58.236518 26219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:38:58.236763 26219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:38:58.237916 26219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:38:58.239431 26219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:38:58.239539 26219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:58.239621 26219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:38:58.239684 26219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:38:58.279871 26219 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:37663
I20260806 19:38:58.279922 26350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:37663 every 8 connection(s)
I20260806 19:38:58.281849 26219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:38:58.287740 26353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:58.285556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:36853)
0806 19:38:58.286477 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:58.286731 (+ 254us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":385,"threads_started":1}
W20260806 19:38:58.288640 26351 heartbeater.cc:644] Failed to heartbeat to 127.10.25.62:36513 (0 consecutive failures): Network error: Failed to ping master at 127.10.25.62:36513: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
I20260806 19:38:58.289690 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 26219
I20260806 19:38:58.289770 26353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:58.289288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:37373)
0806 19:38:58.289519 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:58.289638 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 19:38:58.290793 26353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:58.290389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:36865)
0806 19:38:58.290555 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:58.290658 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 19:38:58.291047 26351 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 19:38:58.687847 26356 raft_consensus.cc:482] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:38:58.688237 26356 raft_consensus.cc:504] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:38:58.690231 26356 leader_election.cc:288] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813), b8da86666b7d4778807c51842118d38c (127.10.25.3:37663)
Aug 06 19:38:58 dist-test-slave-ss4n krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045137, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
Aug 06 19:38:58 dist-test-slave-ss4n krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045137, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:38:58.725978 26240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:58.690456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:45813 (local address 127.10.25.1:57519)
0806 19:38:58.690679 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:58.690733 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:58.691483 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:58.696834 (+ 5351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:58.696846 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:58.697932 (+ 1086us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:58.698896 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:58.698926 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:58.701854 (+ 2928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:58.701862 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:58.704547 (+ 2685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:58.704584 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:58.706859 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:58.706866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:58.707877 (+ 1011us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:58.709461 (+ 1584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:58.709544 (+ 83us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:58.713544 (+ 4000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:58.719636 (+ 6092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:58.719654 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:58.719684 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:58.721158 (+ 1474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:58.722085 (+ 927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:58.722094 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:58.722098 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:58.722546 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:58.723952 (+ 1406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:58.723957 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:58.724440 (+ 483us) client_negotiation.cc:769] Sending connection context
0806 19:38:58.724787 (+ 347us) client_negotiation.cc:240] Negotiation successful
0806 19:38:58.725467 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":569,"spinlock_wait_cycles":67456}
I20260806 19:38:58.725979 26359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:58.691379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:43357 (local address 127.10.25.3:37663)
0806 19:38:58.693456 (+ 2077us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:58.693513 (+ 57us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:58.693562 (+ 49us) server_negotiation.cc:407] Connection header received
0806 19:38:58.696008 (+ 2446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:58.696020 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:58.696517 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:58.696902 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:58.699210 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:58.701530 (+ 2320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:58.704855 (+ 3325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:58.706260 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:58.706531 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:58.714055 (+ 7524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:58.714067 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:58.714080 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:58.714130 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:58.719184 (+ 5054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:58.721435 (+ 2251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:58.721448 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:58.721459 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:58.721853 (+ 394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:58.722729 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:58.722735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:58.722738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:58.723637 (+ 899us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:58.723855 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:58.725043 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0806 19:38:58.725512 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1864,"thread_start_us":1654,"threads_started":1}
I20260806 19:38:58.726068 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:58.690742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:57519 (local address 127.10.25.2:45813)
0806 19:38:58.692605 (+ 1863us) server_negotiation.cc:206] Beginning negotiation
0806 19:38:58.692632 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 19:38:58.692687 (+ 55us) server_negotiation.cc:407] Connection header received
0806 19:38:58.695470 (+ 2783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:38:58.695485 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:38:58.696137 (+ 652us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:38:58.697292 (+ 1155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:38:58.699210 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:58.701530 (+ 2320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:58.704854 (+ 3324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:38:58.706261 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:38:58.706531 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:58.714029 (+ 7498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:38:58.714045 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:38:58.714068 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:38:58.714130 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:38:58.719188 (+ 5058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:58.721435 (+ 2247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:58.721448 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:58.721459 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:58.721853 (+ 394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:38:58.722729 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:38:58.722735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:38:58.722738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:38:58.723664 (+ 926us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:38:58.723855 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 19:38:58.725082 (+ 1227us) server_negotiation.cc:299] Negotiation successful
0806 19:38:58.725555 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1577,"thread_start_us":1406,"threads_started":1}
I20260806 19:38:58.726191 26358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:58.690901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:37663 (local address 127.10.25.1:43357)
0806 19:38:58.693076 (+ 2175us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:58.693110 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 19:38:58.693261 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:38:58.696876 (+ 3615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:38:58.696884 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:38:58.698400 (+ 1516us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:38:58.698896 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:58.698927 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:58.701854 (+ 2927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:58.701862 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:58.704547 (+ 2685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:38:58.704584 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:38:58.706842 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:38:58.706859 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:38:58.707877 (+ 1018us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:38:58.709461 (+ 1584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:38:58.709582 (+ 121us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:38:58.713522 (+ 3940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:38:58.719636 (+ 6114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:58.719654 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:58.719725 (+ 71us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:58.721158 (+ 1433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:58.722085 (+ 927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:38:58.722093 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:38:58.722098 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:38:58.722546 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:38:58.723862 (+ 1316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:38:58.723872 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:38:58.724441 (+ 569us) client_negotiation.cc:769] Sending connection context
0806 19:38:58.724787 (+ 346us) client_negotiation.cc:240] Negotiation successful
0806 19:38:58.725699 (+ 912us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1978,"mutex_wait_us":740,"spinlock_wait_cycles":54656,"thread_start_us":1741,"threads_started":1}
W20260806 19:38:58.731130 25965 leader_election.cc:341] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813): Not found: Tablet not found: 1c3cb447dae54220a81eeb25a57d485c
W20260806 19:38:58.731464 25965 leader_election.cc:341] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8da86666b7d4778807c51842118d38c (127.10.25.3:37663): Not found: Tablet not found: 1c3cb447dae54220a81eeb25a57d485c
I20260806 19:38:58.731710 25965 leader_election.cc:302] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17dc696ed80d4e52beb51b7a6fa9cdac; no voters: 90058d5c0bcb4f42a4b17d02706b5a6e, b8da86666b7d4778807c51842118d38c
I20260806 19:38:58.732163 26356 raft_consensus.cc:2726] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 19:38:58.756050 26360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:58.754884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:39225)
0806 19:38:58.755559 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:58.755749 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":320,"threads_started":1}
I20260806 19:38:59.238096 26361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:59.236955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:35017)
0806 19:38:59.237715 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:59.237833 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":364,"threads_started":1}
I20260806 19:38:59.292805 26362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:59.291842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:43933)
0806 19:38:59.292412 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:59.292563 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":283,"threads_started":1}
I20260806 19:38:59.758399 26363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:38:59.757466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:35737)
0806 19:38:59.758119 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0806 19:38:59.758222 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":304,"threads_started":1}
I20260806 19:39:00.240356 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:00.239402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:48927)
0806 19:39:00.240015 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:00.240122 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":311,"threads_started":1}
I20260806 19:39:00.295601 26365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:00.294217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:51693)
0806 19:39:00.294963 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:00.295116 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":365,"threads_started":1}
I20260806 19:39:00.618002 26366 raft_consensus.cc:482] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:39:00.618232 26366 raft_consensus.cc:504] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:39:00.619115 26366 leader_election.cc:288] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813), b8da86666b7d4778807c51842118d38c (127.10.25.3:37663)
W20260806 19:39:00.620287 25965 leader_election.cc:341] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8da86666b7d4778807c51842118d38c (127.10.25.3:37663): Not found: Tablet not found: 1c3cb447dae54220a81eeb25a57d485c
W20260806 19:39:00.620536 25965 leader_election.cc:341] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813): Not found: Tablet not found: 1c3cb447dae54220a81eeb25a57d485c
I20260806 19:39:00.620702 25965 leader_election.cc:302] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17dc696ed80d4e52beb51b7a6fa9cdac; no voters: 90058d5c0bcb4f42a4b17d02706b5a6e, b8da86666b7d4778807c51842118d38c
I20260806 19:39:00.621090 26366 raft_consensus.cc:2726] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 19:39:00.760612 26367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:00.759657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:58341)
0806 19:39:00.760325 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:00.760422 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":370,"threads_started":1}
I20260806 19:39:01.242770 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:01.241947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:58105)
0806 19:39:01.242458 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:01.242547 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":268,"threads_started":1}
I20260806 19:39:01.298367 26369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:01.297224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:37413)
0806 19:39:01.298008 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:01.298160 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":456,"threads_started":1}
I20260806 19:39:01.763782 26370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:01.762266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:42095)
0806 19:39:01.763317 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:01.763531 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":600,"threads_started":1}
I20260806 19:39:02.246248 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:02.244442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:37573)
0806 19:39:02.245725 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:02.245918 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1008,"thread_start_us":516,"threads_started":1}
I20260806 19:39:02.300693 26372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:02.299635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:56941)
0806 19:39:02.300360 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:02.300466 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":444,"threads_started":1}
I20260806 19:39:02.766031 26373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:02.765118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:35979)
0806 19:39:02.765761 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:02.765846 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":285,"threads_started":1}
I20260806 19:39:03.248561 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:03.247701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:41089)
0806 19:39:03.248247 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:03.248368 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":297,"threads_started":1}
I20260806 19:39:03.280066 26375 raft_consensus.cc:482] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:39:03.280303 26375 raft_consensus.cc:504] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:39:03.281037 26375 leader_election.cc:288] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813), b8da86666b7d4778807c51842118d38c (127.10.25.3:37663)
W20260806 19:39:03.282122 25965 leader_election.cc:341] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813): Not found: Tablet not found: 1c3cb447dae54220a81eeb25a57d485c
W20260806 19:39:03.282373 25965 leader_election.cc:341] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8da86666b7d4778807c51842118d38c (127.10.25.3:37663): Not found: Tablet not found: 1c3cb447dae54220a81eeb25a57d485c
I20260806 19:39:03.282501 25965 leader_election.cc:302] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17dc696ed80d4e52beb51b7a6fa9cdac; no voters: 90058d5c0bcb4f42a4b17d02706b5a6e, b8da86666b7d4778807c51842118d38c
I20260806 19:39:03.282788 26375 raft_consensus.cc:2726] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 19:39:03.302755 26376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:03.301998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:59979)
0806 19:39:03.302474 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:03.302570 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":259,"threads_started":1}
I20260806 19:39:03.768213 26377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:03.767269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:49955)
0806 19:39:03.767891 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:03.767988 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":378,"threads_started":1}
I20260806 19:39:04.250926 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:04.249963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:40561)
0806 19:39:04.250583 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:04.250682 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":366,"threads_started":1}
I20260806 19:39:04.304621 26379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:04.303799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:60059)
0806 19:39:04.304315 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:04.304405 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":277,"threads_started":1}
I20260806 19:39:04.770659 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:04.769685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:46301)
0806 19:39:04.770348 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:04.770439 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":305,"threads_started":1}
I20260806 19:39:05.182621 26381 raft_consensus.cc:482] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:39:05.182881 26381 raft_consensus.cc:504] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:39:05.183610 26381 leader_election.cc:288] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813), b8da86666b7d4778807c51842118d38c (127.10.25.3:37663)
W20260806 19:39:05.185289 25965 leader_election.cc:341] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813): Not found: Tablet not found: 1c3cb447dae54220a81eeb25a57d485c
W20260806 19:39:05.185789 25965 leader_election.cc:341] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8da86666b7d4778807c51842118d38c (127.10.25.3:37663): Not found: Tablet not found: 1c3cb447dae54220a81eeb25a57d485c
I20260806 19:39:05.185896 25965 leader_election.cc:302] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17dc696ed80d4e52beb51b7a6fa9cdac; no voters: 90058d5c0bcb4f42a4b17d02706b5a6e, b8da86666b7d4778807c51842118d38c
I20260806 19:39:05.186164 26381 raft_consensus.cc:2726] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 19:39:05.253284 26382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:05.252154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:41153)
0806 19:39:05.252938 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:05.253036 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":408,"threads_started":1}
I20260806 19:39:05.307453 26383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:05.306135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:60889)
0806 19:39:05.307098 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:05.307206 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":457,"threads_started":1}
I20260806 19:39:05.772719 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:05.771813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:53705)
0806 19:39:05.772373 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:05.772467 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":277,"threads_started":1}
I20260806 19:39:06.255751 26385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:06.254643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:43099)
0806 19:39:06.255357 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:06.255473 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":335,"threads_started":1}
I20260806 19:39:06.309654 26386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:06.308726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:50159)
0806 19:39:06.309246 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:06.309339 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":283,"threads_started":1}
I20260806 19:39:06.774804 26387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:06.773887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:35809)
0806 19:39:06.774477 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:06.774601 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":290,"threads_started":1}
I20260806 19:39:07.258440 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:07.257038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:55113)
0806 19:39:07.257947 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:07.258152 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":548,"threads_started":1}
I20260806 19:39:07.311750 26389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:07.310853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:40399)
0806 19:39:07.311457 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:07.311565 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":302,"threads_started":1}
I20260806 19:39:07.680254 26390 raft_consensus.cc:482] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:39:07.680482 26390 raft_consensus.cc:504] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17dc696ed80d4e52beb51b7a6fa9cdac" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 34819 } } peers { permanent_uuid: "90058d5c0bcb4f42a4b17d02706b5a6e" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45813 } } peers { permanent_uuid: "b8da86666b7d4778807c51842118d38c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 37663 } }
I20260806 19:39:07.681340 26390 leader_election.cc:288] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813), b8da86666b7d4778807c51842118d38c (127.10.25.3:37663)
W20260806 19:39:07.683346 25965 leader_election.cc:341] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8da86666b7d4778807c51842118d38c (127.10.25.3:37663): Not found: Tablet not found: 1c3cb447dae54220a81eeb25a57d485c
W20260806 19:39:07.683760 25965 leader_election.cc:341] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90058d5c0bcb4f42a4b17d02706b5a6e (127.10.25.2:45813): Not found: Tablet not found: 1c3cb447dae54220a81eeb25a57d485c
I20260806 19:39:07.683894 25965 leader_election.cc:302] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17dc696ed80d4e52beb51b7a6fa9cdac; no voters: 90058d5c0bcb4f42a4b17d02706b5a6e, b8da86666b7d4778807c51842118d38c
I20260806 19:39:07.684217 26390 raft_consensus.cc:2726] T 1c3cb447dae54220a81eeb25a57d485c P 17dc696ed80d4e52beb51b7a6fa9cdac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 19:39:07.777284 26391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:07.776185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:59079)
0806 19:39:07.776819 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:07.776920 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":344,"threads_started":1}
I20260806 19:39:08.260841 26392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:08.259649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:35855)
0806 19:39:08.260418 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:08.260579 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":394,"threads_started":1}
I20260806 19:39:08.313994 26393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:08.313063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:59457)
0806 19:39:08.313699 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:08.313806 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":331,"threads_started":1}
I20260806 19:39:08.779837 26394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:08.778860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:47201)
0806 19:39:08.779469 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:08.779603 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":302,"threads_started":1}
I20260806 19:39:09.263600 26395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:09.262298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:57179)
0806 19:39:09.263188 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:09.263366 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":438,"threads_started":1}
I20260806 19:39:09.316540 26396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:09.315448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:43197)
0806 19:39:09.316202 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:09.316324 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":373,"threads_started":1}
I20260806 19:39:09.782287 26397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:09.781225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:36925)
0806 19:39:09.781952 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:09.782057 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":400,"threads_started":1}
I20260806 19:39:10.265769 26398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:10.264810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:54881)
0806 19:39:10.265362 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:10.265536 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":318,"threads_started":1}
I20260806 19:39:10.319314 26399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:10.318103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:42719)
0806 19:39:10.318992 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:10.319098 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":628,"threads_started":1}
I20260806 19:39:10.784659 26400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:10.783743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:48379)
0806 19:39:10.784322 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:10.784435 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":294,"threads_started":1}
I20260806 19:39:11.268466 26401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:11.267428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:39575)
0806 19:39:11.268168 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:11.268260 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":347,"threads_started":1}
I20260806 19:39:11.321969 26402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:11.320628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:44511)
0806 19:39:11.321662 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:11.321783 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":516,"threads_started":1}
I20260806 19:39:11.787485 26403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:11.786231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:59909)
0806 19:39:11.787104 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:11.787236 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":380,"threads_started":1}
I20260806 19:39:12.271113 26404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:12.270153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:52773)
0806 19:39:12.270783 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:12.270890 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":390,"threads_started":1}
I20260806 19:39:12.324339 26405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:12.323463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:32835)
0806 19:39:12.324011 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:12.324111 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":274,"threads_started":1}
I20260806 19:39:12.790323 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:12.789101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.2:59003)
0806 19:39:12.789972 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:12.790117 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":365,"threads_started":1}
I20260806 19:39:13.273320 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:13.272413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.1:51925)
0806 19:39:13.272977 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:13.273074 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":300,"threads_started":1}
I20260806 19:39:13.301687 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 25909
I20260806 19:39:13.322088 26408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:13.303435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42148 (local address 127.10.25.1:34819)
0806 19:39:13.304157 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:13.304166 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:13.304202 (+ 36us) server_negotiation.cc:407] Connection header received
0806 19:39:13.304285 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:13.304293 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:13.304640 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:13.304867 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:13.307041 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:13.308278 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:13.309937 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:13.310702 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:13.310832 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:13.314077 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:13.314094 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:13.314122 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:13.314173 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:13.318097 (+ 3924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:13.319345 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:13.319356 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:13.319359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:13.319605 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:13.320145 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:13.320150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:13.320152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:13.320579 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:13.320955 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:13.321294 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 19:39:13.321678 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":293,"threads_started":1}
I20260806 19:39:13.323639 26076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:39:13.327446 26410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:13.326350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36513 (local address 127.10.25.3:34645)
0806 19:39:13.327009 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:13.327182 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:36513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":317,"threads_started":1}
I20260806 19:39:13.430549 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 25909
I20260806 19:39:13.452636 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 26084
I20260806 19:39:13.471450 26411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:13.453624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51048 (local address 127.10.25.2:45813)
0806 19:39:13.454177 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:13.454182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:13.454199 (+ 17us) server_negotiation.cc:407] Connection header received
0806 19:39:13.454258 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:13.454262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:13.454324 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:13.454456 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:13.455946 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:13.456961 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:13.458661 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:13.459205 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:13.459381 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:13.462930 (+ 3549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:13.462937 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:13.462940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:13.462996 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:13.466831 (+ 3835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:13.468339 (+ 1508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:13.468346 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:13.468349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:13.468588 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:13.469225 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:13.469230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:13.469232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:13.469697 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:13.470113 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:13.470365 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 19:39:13.470716 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":302,"threads_started":1}
I20260806 19:39:13.472664 26211 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:39:13.564863 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 26084
I20260806 19:39:13.586102 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 26219
I20260806 19:39:13.605154 26413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:13.587616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43354 (local address 127.10.25.3:37663)
0806 19:39:13.588366 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:13.588371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:13.588390 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:39:13.588455 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:13.588458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:13.588547 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:13.588702 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:13.590589 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:13.591572 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:13.593162 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:13.593765 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:13.593939 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:13.598219 (+ 4280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:13.598226 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:13.598230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:13.598283 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:13.602014 (+ 3731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:13.602960 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:13.602967 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:13.602970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:13.603135 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:13.603545 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:13.603550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:13.603553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:13.603854 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:13.604180 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:13.604478 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 19:39:13.604743 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":400,"threads_started":1}
I20260806 19:39:13.606069 26346 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:39:13.702046 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 26219
2026-08-06T19:39:13Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (26074 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:39:13 dist-test-slave-ss4n krb5kdc[26417](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:39:13 dist-test-slave-ss4n krb5kdc[26417](info): set up 2 sockets
Aug 06 19:39:13 dist-test-slave-ss4n krb5kdc[26417](info): commencing operation
krb5kdc: starting...
W20260806 19:39:14.965730 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.015s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:39:15 dist-test-slave-ss4n krb5kdc[26417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:39:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:39:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:39:15.303470 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:43233
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:43233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:39:15.713356 26430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:39:15.713815 26430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:39:15.714018 26430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:39:15.722489 26430 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 19:39:15.722733 26430 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 19:39:15.722862 26430 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 19:39:15.722975 26430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:39:15.723071 26430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:39:15.723197 26430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:39:15.723299 26430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:39:15.723397 26430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:39:15.734427 26430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:43233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:43233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:39:15.735913 26430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:39:15.737994 26430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:39:15.756042 26435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:39:15.756170 26436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:39:15.756184 26438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:39:15.758054 26430 server_base.cc:1034] running on GCE node
I20260806 19:39:15.759153 26430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:39:15.761042 26430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:39:15.762360 26430 hybrid_clock.cc:648] HybridClock initialized: now 1786045155762231 us; error 95 us; skew 500 ppm
I20260806 19:39:15.762849 26430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:39:15 dist-test-slave-ss4n krb5kdc[26417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045155, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:39:15.772960 26430 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:39:15.775122 26430 webserver.cc:460] Webserver started at http://127.10.25.62:36495/ using document root <none> and password file <none>
I20260806 19:39:15.775990 26430 fs_manager.cc:359] Metadata directory not provided
I20260806 19:39:15.776124 26430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:39:15.776520 26430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:39:15.780032 26430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "9d61ccd09113402aa039f1960b2dc3a6"
format_stamp: "Formatted at 2026-08-06 19:39:15 on dist-test-slave-ss4n"
server_key: "db23c1de69d2ef087708a7a5d487d557"
server_key_iv: "042c540f263c0edf0c53049dc0757b21"
server_key_version: "encryptionkey@0"
I20260806 19:39:15.780982 26430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "9d61ccd09113402aa039f1960b2dc3a6"
format_stamp: "Formatted at 2026-08-06 19:39:15 on dist-test-slave-ss4n"
server_key: "db23c1de69d2ef087708a7a5d487d557"
server_key_iv: "042c540f263c0edf0c53049dc0757b21"
server_key_version: "encryptionkey@0"
I20260806 19:39:15.787299 26430 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 19:39:15.791607 26445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:15.793329 26430 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 19:39:15.793684 26430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "9d61ccd09113402aa039f1960b2dc3a6"
format_stamp: "Formatted at 2026-08-06 19:39:15 on dist-test-slave-ss4n"
server_key: "db23c1de69d2ef087708a7a5d487d557"
server_key_iv: "042c540f263c0edf0c53049dc0757b21"
server_key_version: "encryptionkey@0"
I20260806 19:39:15.793943 26430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:39:15.827687 26430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:39:15.833670 26430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:39:15.833967 26430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:39:15.855480 26430 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:43233
I20260806 19:39:15.855525 26498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:43233 every 8 connection(s)
I20260806 19:39:15.857010 26430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:39:15.860970 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 26430
I20260806 19:39:15.861253 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:39:15.861594 26499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:39:15.862144 10340 external_mini_cluster.cc:1428] Setting key f109ebf443f8c5225d228d8ffeadff7d
I20260806 19:39:15.876394 26499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Bootstrap starting.
Aug 06 19:39:15 dist-test-slave-ss4n krb5kdc[26417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:39:15.880266 26499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Neither blocks nor log segments found. Creating new log.
I20260806 19:39:15.881747 26499 log.cc:824] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Log is configured to *not* fsync() on all Append() calls
I20260806 19:39:15.885257 26499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: No bootstrap required, opened a new log
I20260806 19:39:15.892007 26502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:15.864365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35144 (local address 127.10.25.62:43233)
0806 19:39:15.865708 (+ 1343us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:15.865724 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:15.865785 (+ 61us) server_negotiation.cc:407] Connection header received
0806 19:39:15.867557 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:15.867586 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:15.868122 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:15.868624 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:15.870650 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:15.872074 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:15.873838 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:15.874759 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:15.875020 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:15.880512 (+ 5492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:15.880524 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:15.880543 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:15.880595 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:15.884559 (+ 3964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:15.885638 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:15.885648 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:15.885658 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:15.885939 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:15.886679 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:15.886685 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:15.886689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:15.888163 (+ 1474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:15.888580 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:15.889300 (+ 720us) server_negotiation.cc:299] Negotiation successful
0806 19:39:15.889951 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":845,"thread_start_us":451,"threads_started":1}
I20260806 19:39:15.899232 26499 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } }
I20260806 19:39:15.899705 26499 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:39:15.899792 26499 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d61ccd09113402aa039f1960b2dc3a6, State: Initialized, Role: FOLLOWER
I20260806 19:39:15.900362 26499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } }
I20260806 19:39:15.900612 26499 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:39:15.900717 26499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:39:15.900856 26499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:39:15.903398 26499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } }
I20260806 19:39:15.904083 26499 leader_election.cc:302] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d61ccd09113402aa039f1960b2dc3a6; no voters:
I20260806 19:39:15.904827 26499 leader_election.cc:288] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:39:15.905007 26504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:39:15.905772 26504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 1 LEADER]: Becoming Leader. State: Replica: 9d61ccd09113402aa039f1960b2dc3a6, State: Running, Role: LEADER
I20260806 19:39:15.906483 26504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } }
I20260806 19:39:15.907361 26499 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:39:15.911273 26505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d61ccd09113402aa039f1960b2dc3a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } } }
I20260806 19:39:15.911923 26505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [sys.catalog]: This master's current role is: LEADER
I20260806 19:39:15.915125 26506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d61ccd09113402aa039f1960b2dc3a6. Latest consensus state: current_term: 1 leader_uuid: "9d61ccd09113402aa039f1960b2dc3a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } } }
I20260806 19:39:15.916045 26506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [sys.catalog]: This master's current role is: LEADER
I20260806 19:39:15.919917 26513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:39:15.933400 26513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:39:15.945683 26513 catalog_manager.cc:1347] Generated new cluster ID: eb8501212f7b48e4a6bfe0d15f50defe
I20260806 19:39:15.945930 26513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:39:15.961108 26513 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:39:15.963248 26513 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:39:15.977591 26513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Generated new TSK 0
I20260806 19:39:15.979082 26513 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:39:16.080106 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:43233
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:39:16.523653 26525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:39:16.524019 26525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:39:16.524137 26525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:39:16.533821 26525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:39:16.534031 26525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:39:16.534180 26525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:39:16.547401 26525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:43233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:39:16.549031 26525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:39:16.550625 26525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:39:16.567433 26530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:39:16.567584 26531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:39:16.567762 26533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:39:16.569701 26525 server_base.cc:1034] running on GCE node
I20260806 19:39:16.570657 26525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:39:16.572314 26525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:39:16.573875 26525 hybrid_clock.cc:648] HybridClock initialized: now 1786045156573814 us; error 112 us; skew 500 ppm
I20260806 19:39:16.574421 26525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:39:16 dist-test-slave-ss4n krb5kdc[26417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045156, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:39:16.579656 26525 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:39:16.582579 26525 webserver.cc:460] Webserver started at http://127.10.25.1:36833/ using document root <none> and password file <none>
I20260806 19:39:16.584018 26525 fs_manager.cc:359] Metadata directory not provided
I20260806 19:39:16.584214 26525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:39:16.584633 26525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:39:16.588706 26525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "9da9351acb7e41539f7b2f0ff4da3aa7"
format_stamp: "Formatted at 2026-08-06 19:39:16 on dist-test-slave-ss4n"
server_key: "8b85c43e172ff9ac7c1a0c8c07193e2c"
server_key_iv: "fee25853912f5dbc3ed52b790e402fd8"
server_key_version: "encryptionkey@0"
I20260806 19:39:16.589902 26525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "9da9351acb7e41539f7b2f0ff4da3aa7"
format_stamp: "Formatted at 2026-08-06 19:39:16 on dist-test-slave-ss4n"
server_key: "8b85c43e172ff9ac7c1a0c8c07193e2c"
server_key_iv: "fee25853912f5dbc3ed52b790e402fd8"
server_key_version: "encryptionkey@0"
I20260806 19:39:16.597380 26525 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260806 19:39:16.603477 26540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:16.606122 26525 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.001s
I20260806 19:39:16.606510 26525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "9da9351acb7e41539f7b2f0ff4da3aa7"
format_stamp: "Formatted at 2026-08-06 19:39:16 on dist-test-slave-ss4n"
server_key: "8b85c43e172ff9ac7c1a0c8c07193e2c"
server_key_iv: "fee25853912f5dbc3ed52b790e402fd8"
server_key_version: "encryptionkey@0"
I20260806 19:39:16.606869 26525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:39:16.632886 26525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:39:16.640748 26525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:39:16.641237 26525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:39:16.643081 26525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:39:16.645131 26525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:39:16.645292 26525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:16.645545 26525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:39:16.645649 26525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 19:39:16.696280 26525 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:45971
I20260806 19:39:16.696336 26654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:45971 every 8 connection(s)
I20260806 19:39:16.698086 26525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:39:16.707362 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 26525
I20260806 19:39:16.707711 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:39:16.709208 10340 external_mini_cluster.cc:1428] Setting key a1afee143d05d386563026a62d331406
Aug 06 19:39:16 dist-test-slave-ss4n krb5kdc[26417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045156, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:39:16.728484 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:16.702048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:48737 (local address 127.10.25.62:43233)
0806 19:39:16.702734 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:16.702739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:16.704112 (+ 1373us) server_negotiation.cc:407] Connection header received
0806 19:39:16.706513 (+ 2401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:16.706520 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:16.706639 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:16.706783 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:16.709006 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:16.710030 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:16.712162 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:16.712669 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:16.712854 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:16.718587 (+ 5733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:16.718597 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:16.718605 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:16.718668 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:16.723760 (+ 5092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:16.725229 (+ 1469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:16.725239 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:16.725246 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:16.725462 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:16.726136 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:16.726144 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:16.726149 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:16.726644 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:16.726819 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:16.727935 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0806 19:39:16.728193 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":398,"threads_started":1}
I20260806 19:39:16.731184 26658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:16.702787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:48737)
0806 19:39:16.703849 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:16.703910 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:16.706130 (+ 2220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:16.707102 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:16.707113 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:16.707953 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:39:16.708743 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:16.708767 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:16.710246 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:16.710252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:16.711822 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:16.711847 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:16.713006 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:16.713015 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:16.713340 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:16.714405 (+ 1065us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:39:16.714439 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:39:16.718182 (+ 3743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:39:16.724128 (+ 5946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:39:16.724251 (+ 123us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:39:16.724283 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:39:16.725007 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:39:16.725712 (+ 705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:39:16.725716 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:39:16.725719 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:39:16.725985 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:39:16.726859 (+ 874us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:39:16.726867 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:39:16.727400 (+ 533us) client_negotiation.cc:769] Sending connection context
0806 19:39:16.727828 (+ 428us) client_negotiation.cc:240] Negotiation successful
0806 19:39:16.728749 (+ 921us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"thread_start_us":448,"threads_started":1}
I20260806 19:39:16.733649 26655 heartbeater.cc:347] Connected to a master server at 127.10.25.62:43233
I20260806 19:39:16.734093 26655 heartbeater.cc:464] Registering TS with master...
I20260806 19:39:16.735004 26655 heartbeater.cc:511] Master 127.10.25.62:43233 requested a full tablet report, sending...
I20260806 19:39:16.738195 26464 ts_manager.cc:194] Registered new tserver with Master: 9da9351acb7e41539f7b2f0ff4da3aa7 (127.10.25.1:45971)
I20260806 19:39:16.740139 26464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:48737
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:39:16.795380 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:43233
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:39:17.230654 26661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:39:17.231031 26661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:39:17.231233 26661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:39:17.239071 26661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:39:17.239275 26661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:39:17.239454 26661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:39:17.250339 26661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:43233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:39:17.251843 26661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:39:17.253397 26661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:39:17.271073 26669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:39:17.271133 26667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:39:17.271178 26666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:39:17.273265 26661 server_base.cc:1034] running on GCE node
I20260806 19:39:17.274247 26661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:39:17.275607 26661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:39:17.276925 26661 hybrid_clock.cc:648] HybridClock initialized: now 1786045157276870 us; error 55 us; skew 500 ppm
I20260806 19:39:17.277419 26661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:39:17 dist-test-slave-ss4n krb5kdc[26417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045157, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:39:17.281805 26661 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:39:17.284305 26661 webserver.cc:460] Webserver started at http://127.10.25.2:46557/ using document root <none> and password file <none>
I20260806 19:39:17.285202 26661 fs_manager.cc:359] Metadata directory not provided
I20260806 19:39:17.285387 26661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:39:17.285849 26661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:39:17.289281 26661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "5dc359d524bc4759845f6781e9f6738d"
format_stamp: "Formatted at 2026-08-06 19:39:17 on dist-test-slave-ss4n"
server_key: "45cffeac6f03a84ff047ad823c0b0e6d"
server_key_iv: "b855018a4064c41afe4f9f2ad64d95e8"
server_key_version: "encryptionkey@0"
I20260806 19:39:17.290359 26661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "5dc359d524bc4759845f6781e9f6738d"
format_stamp: "Formatted at 2026-08-06 19:39:17 on dist-test-slave-ss4n"
server_key: "45cffeac6f03a84ff047ad823c0b0e6d"
server_key_iv: "b855018a4064c41afe4f9f2ad64d95e8"
server_key_version: "encryptionkey@0"
I20260806 19:39:17.297103 26661 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:39:17.303315 26676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:17.305229 26661 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260806 19:39:17.305552 26661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "5dc359d524bc4759845f6781e9f6738d"
format_stamp: "Formatted at 2026-08-06 19:39:17 on dist-test-slave-ss4n"
server_key: "45cffeac6f03a84ff047ad823c0b0e6d"
server_key_iv: "b855018a4064c41afe4f9f2ad64d95e8"
server_key_version: "encryptionkey@0"
I20260806 19:39:17.305874 26661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:39:17.324582 26661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:39:17.330173 26661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:39:17.330646 26661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:39:17.331872 26661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:39:17.333575 26661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:39:17.333706 26661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:17.333879 26661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:39:17.333961 26661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:17.372573 26661 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:41159
I20260806 19:39:17.372711 26788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:41159 every 8 connection(s)
I20260806 19:39:17.374151 26661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:39:17.375443 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 26661
I20260806 19:39:17.375679 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:39:17.376164 10340 external_mini_cluster.cc:1428] Setting key 6fe5d48645298265da6d87a816212447
Aug 06 19:39:17 dist-test-slave-ss4n krb5kdc[26417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045157, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:39:17.400152 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:17.377561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:37057 (local address 127.10.25.62:43233)
0806 19:39:17.378084 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:17.378090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:17.379285 (+ 1195us) server_negotiation.cc:407] Connection header received
0806 19:39:17.381445 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:17.381451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:17.381601 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:17.381741 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:17.383695 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:17.384691 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:17.386904 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:17.387417 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:17.387603 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:17.392720 (+ 5117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:17.392728 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:17.392732 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:17.392774 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:17.396070 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:17.397117 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:17.397123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:17.397125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:17.397280 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:17.397918 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:17.397928 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:17.397931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:17.398260 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:17.398450 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:17.399369 (+ 919us) server_negotiation.cc:299] Negotiation successful
0806 19:39:17.399684 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":257,"threads_started":1}
I20260806 19:39:17.401341 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:17.378162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:37057)
0806 19:39:17.379090 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:17.379143 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:17.381104 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:17.382043 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:17.382060 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:17.382729 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:39:17.383434 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:17.383457 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:17.384916 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:17.384921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:17.386701 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:17.386719 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:17.387731 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:17.387739 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:17.388025 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:17.388907 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:39:17.388934 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:39:17.392339 (+ 3405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:39:17.396320 (+ 3981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:39:17.396332 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:39:17.396362 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:39:17.396926 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:39:17.397432 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:39:17.397439 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:39:17.397443 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:39:17.397765 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:39:17.398420 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:39:17.398430 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:39:17.398895 (+ 465us) client_negotiation.cc:769] Sending connection context
0806 19:39:17.399215 (+ 320us) client_negotiation.cc:240] Negotiation successful
0806 19:39:17.399988 (+ 773us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":352,"threads_started":1}
I20260806 19:39:17.403136 26789 heartbeater.cc:347] Connected to a master server at 127.10.25.62:43233
I20260806 19:39:17.403524 26789 heartbeater.cc:464] Registering TS with master...
I20260806 19:39:17.404392 26789 heartbeater.cc:511] Master 127.10.25.62:43233 requested a full tablet report, sending...
I20260806 19:39:17.406246 26464 ts_manager.cc:194] Registered new tserver with Master: 5dc359d524bc4759845f6781e9f6738d (127.10.25.2:41159)
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
I20260806 19:39:17.407323 26464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:37057
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:39:17.451761 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:43233
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:39:17.744303 26655 heartbeater.cc:503] Master 127.10.25.62:43233 was elected leader, sending a full tablet report...
W20260806 19:39:17.854415 26795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:39:17.854857 26795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:39:17.855059 26795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:39:17.863725 26795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:39:17.863919 26795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:39:17.864104 26795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:39:17.875097 26795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:43233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:39:17.876616 26795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:39:17.878191 26795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:39:17.894841 26800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:39:17.894719 26801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:39:17.894814 26803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:39:17.895788 26795 server_base.cc:1034] running on GCE node
I20260806 19:39:17.896612 26795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:39:17.897840 26795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:39:17.899125 26795 hybrid_clock.cc:648] HybridClock initialized: now 1786045157899051 us; error 79 us; skew 500 ppm
I20260806 19:39:17.899602 26795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:39:17 dist-test-slave-ss4n krb5kdc[26417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045157, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:39:17.904851 26795 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:39:17.906920 26795 webserver.cc:460] Webserver started at http://127.10.25.3:40743/ using document root <none> and password file <none>
I20260806 19:39:17.907837 26795 fs_manager.cc:359] Metadata directory not provided
I20260806 19:39:17.908015 26795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:39:17.908457 26795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:39:17.912360 26795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "26a908fd090745318dda758213a0acb1"
format_stamp: "Formatted at 2026-08-06 19:39:17 on dist-test-slave-ss4n"
server_key: "dfb314d0f2acc657b313e0e6ce440d49"
server_key_iv: "f485b6ad44a4bd58ce2643ab73c0dba9"
server_key_version: "encryptionkey@0"
I20260806 19:39:17.913520 26795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "26a908fd090745318dda758213a0acb1"
format_stamp: "Formatted at 2026-08-06 19:39:17 on dist-test-slave-ss4n"
server_key: "dfb314d0f2acc657b313e0e6ce440d49"
server_key_iv: "f485b6ad44a4bd58ce2643ab73c0dba9"
server_key_version: "encryptionkey@0"
I20260806 19:39:17.923343 26795 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.001s
I20260806 19:39:17.928757 26810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:17.930842 26795 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260806 19:39:17.931223 26795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "26a908fd090745318dda758213a0acb1"
format_stamp: "Formatted at 2026-08-06 19:39:17 on dist-test-slave-ss4n"
server_key: "dfb314d0f2acc657b313e0e6ce440d49"
server_key_iv: "f485b6ad44a4bd58ce2643ab73c0dba9"
server_key_version: "encryptionkey@0"
I20260806 19:39:17.931476 26795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:39:17.971287 26795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:39:17.976728 26795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:39:17.977097 26795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:39:17.978595 26795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:39:17.980329 26795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:39:17.980448 26795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:17.980623 26795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:39:17.980711 26795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:18.020658 26795 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:36477
I20260806 19:39:18.020731 26924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:36477 every 8 connection(s)
I20260806 19:39:18.022297 26795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:39:18.030006 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 26795
I20260806 19:39:18.030249 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:39:18.030726 10340 external_mini_cluster.cc:1428] Setting key f5993efad886ec7d9939cacce46e2763
Aug 06 19:39:18 dist-test-slave-ss4n krb5kdc[26417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045157, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:39:18.047344 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.025446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:43463 (local address 127.10.25.62:43233)
0806 19:39:18.026003 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:18.026008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:18.027230 (+ 1222us) server_negotiation.cc:407] Connection header received
0806 19:39:18.029515 (+ 2285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:18.029522 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:18.029661 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:18.029806 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:18.031878 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.032761 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.034832 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.035313 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.035474 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.040349 (+ 4875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:18.040355 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:18.040358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:18.040398 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:18.043654 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:18.044636 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:18.044641 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:18.044644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:18.044799 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:18.045298 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:18.045308 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:18.045310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:18.045706 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:18.045882 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:18.046659 (+ 777us) server_negotiation.cc:299] Negotiation successful
0806 19:39:18.046915 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":273,"threads_started":1}
I20260806 19:39:18.048408 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.026142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:43463)
0806 19:39:18.027005 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:18.027062 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:18.029112 (+ 2050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:18.030141 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:18.030158 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:18.030944 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:39:18.031653 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:18.031675 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.032973 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.032979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:18.034652 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:18.034669 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.035650 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.035662 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:18.035924 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.037219 (+ 1295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:39:18.037257 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:39:18.040024 (+ 2767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:39:18.043898 (+ 3874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:39:18.043908 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:39:18.043925 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:39:18.044455 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:39:18.044942 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:39:18.044947 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:39:18.044950 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:39:18.045161 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:39:18.045847 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:39:18.045857 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:39:18.046256 (+ 399us) client_negotiation.cc:769] Sending connection context
0806 19:39:18.046557 (+ 301us) client_negotiation.cc:240] Negotiation successful
0806 19:39:18.047156 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":369,"threads_started":1}
I20260806 19:39:18.050189 26925 heartbeater.cc:347] Connected to a master server at 127.10.25.62:43233
I20260806 19:39:18.050562 26925 heartbeater.cc:464] Registering TS with master...
I20260806 19:39:18.051440 26925 heartbeater.cc:511] Master 127.10.25.62:43233 requested a full tablet report, sending...
I20260806 19:39:18.053269 26464 ts_manager.cc:194] Registered new tserver with Master: 26a908fd090745318dda758213a0acb1 (127.10.25.3:36477)
I20260806 19:39:18.054332 26464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:43463
I20260806 19:39:18.060413 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:39:18.083597 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.065832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35156 (local address 127.10.25.62:43233)
0806 19:39:18.066043 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:18.066049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:18.066644 (+ 595us) server_negotiation.cc:407] Connection header received
0806 19:39:18.066798 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:18.066803 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:18.066899 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:18.067011 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:18.069436 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.070378 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.072099 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.072577 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.072745 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.076331 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:18.076338 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:18.076341 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:18.076391 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:18.079702 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:18.080748 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:18.080756 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:18.080759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:18.081062 (+ 303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:18.081839 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:18.081845 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:18.081848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:18.082341 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:18.082585 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:18.083083 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 19:39:18.083335 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 19:39:18.089839 26464 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:39:18.097343 26464 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:39:18.134248 26938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.113810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:36477 (local address 127.0.0.1:34270)
0806 19:39:18.119020 (+ 5210us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:18.119049 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:18.119189 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:18.124343 (+ 5154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:18.124354 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:18.124406 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:39:18.124871 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:18.124885 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.127367 (+ 2482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.127373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:18.130554 (+ 3181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:18.130572 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.132904 (+ 2332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.132917 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:18.133378 (+ 461us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.133415 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 19:39:18.133567 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 19:39:18.133811 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4857,"mutex_wait_us":195,"thread_start_us":986,"threads_started":1}
I20260806 19:39:18.134433 26939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.113567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58582 (local address 127.10.25.1:45971)
0806 19:39:18.124592 (+ 11025us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:18.124602 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:18.124632 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:39:18.124739 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:18.124749 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:18.125029 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:18.125236 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:39:18.126214 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.128171 (+ 1957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.130691 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.132244 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.132402 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.132672 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:18.133773 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0806 19:39:18.134006 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10831,"thread_start_us":10656,"threads_started":1}
I20260806 19:39:18.135275 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.114555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34270 (local address 127.10.25.3:36477)
0806 19:39:18.123492 (+ 8937us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:18.123503 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:18.123560 (+ 57us) server_negotiation.cc:407] Connection header received
0806 19:39:18.123687 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:18.123695 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:18.124017 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:18.125122 (+ 1105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:39:18.125443 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.127199 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.130780 (+ 3581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.132089 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.132278 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.132585 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:18.134570 (+ 1985us) server_negotiation.cc:299] Negotiation successful
0806 19:39:18.134793 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8703,"thread_start_us":9232,"threads_started":1}
I20260806 19:39:18.135283 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.113404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:45971 (local address 127.0.0.1:58582)
0806 19:39:18.114621 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:18.114678 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:18.114878 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:18.125290 (+ 10412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:18.125299 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:18.125346 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:39:18.125964 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:18.125980 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.128350 (+ 2370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.128356 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:18.130484 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:18.130499 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.132955 (+ 2456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.132964 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:18.133287 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.133382 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 19:39:18.133536 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 19:39:18.134818 (+ 1282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"mutex_wait_us":1077,"thread_start_us":4913,"threads_started":1}
I20260806 19:39:18.137696 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.122704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:41159 (local address 127.0.0.1:52576)
0806 19:39:18.126541 (+ 3837us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:18.126559 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:18.126738 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:18.127723 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:18.127727 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:18.127761 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:39:18.129124 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:18.129140 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.131328 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.131334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:18.133809 (+ 2475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:18.133824 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.136635 (+ 2811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.136645 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:18.136869 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.136915 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 19:39:18.137038 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 19:39:18.137217 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3700,"mutex_wait_us":29,"thread_start_us":3476,"threads_started":1}
I20260806 19:39:18.137846 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.119354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52576 (local address 127.10.25.2:41159)
0806 19:39:18.126089 (+ 6735us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:18.126099 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:18.126687 (+ 588us) server_negotiation.cc:407] Connection header received
0806 19:39:18.126925 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:18.126933 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:18.127199 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:18.127425 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:39:18.129388 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.131133 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.134031 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.136029 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.136195 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.136457 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:18.137127 (+ 670us) server_negotiation.cc:299] Negotiation successful
0806 19:39:18.137335 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6540,"thread_start_us":6401,"threads_started":1}
I20260806 19:39:18.139616 26860 tablet_service.cc:1467] Processing CreateTablet for tablet d1eecca4bf1249768a2fb332d0d7f807 (DEFAULT_TABLE table=test-table [id=fefd577442564b1fb74f5ad3cc8e9223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:39:18.139638 26588 tablet_service.cc:1467] Processing CreateTablet for tablet d1eecca4bf1249768a2fb332d0d7f807 (DEFAULT_TABLE table=test-table [id=fefd577442564b1fb74f5ad3cc8e9223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:39:18.140993 26860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1eecca4bf1249768a2fb332d0d7f807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:39:18.141247 26588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1eecca4bf1249768a2fb332d0d7f807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:39:18.142318 26724 tablet_service.cc:1467] Processing CreateTablet for tablet d1eecca4bf1249768a2fb332d0d7f807 (DEFAULT_TABLE table=test-table [id=fefd577442564b1fb74f5ad3cc8e9223]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:39:18.143999 26724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1eecca4bf1249768a2fb332d0d7f807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:39:18.156376 26944 tablet_bootstrap.cc:492] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1: Bootstrap starting.
I20260806 19:39:18.158931 26945 tablet_bootstrap.cc:492] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7: Bootstrap starting.
I20260806 19:39:18.161643 26946 tablet_bootstrap.cc:492] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d: Bootstrap starting.
I20260806 19:39:18.162786 26945 tablet_bootstrap.cc:654] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7: Neither blocks nor log segments found. Creating new log.
I20260806 19:39:18.162766 26944 tablet_bootstrap.cc:654] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1: Neither blocks nor log segments found. Creating new log.
I20260806 19:39:18.164256 26945 log.cc:824] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7: Log is configured to *not* fsync() on all Append() calls
I20260806 19:39:18.164250 26944 log.cc:824] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1: Log is configured to *not* fsync() on all Append() calls
I20260806 19:39:18.165558 26946 tablet_bootstrap.cc:654] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d: Neither blocks nor log segments found. Creating new log.
I20260806 19:39:18.166826 26946 log.cc:824] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d: Log is configured to *not* fsync() on all Append() calls
I20260806 19:39:18.168059 26944 tablet_bootstrap.cc:492] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1: No bootstrap required, opened a new log
I20260806 19:39:18.168057 26945 tablet_bootstrap.cc:492] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7: No bootstrap required, opened a new log
I20260806 19:39:18.168387 26945 ts_tablet_manager.cc:1397] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 19:39:18.168387 26944 ts_tablet_manager.cc:1397] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260806 19:39:18.170010 26946 tablet_bootstrap.cc:492] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d: No bootstrap required, opened a new log
I20260806 19:39:18.170329 26946 ts_tablet_manager.cc:1397] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 19:39:18.178869 26946 raft_consensus.cc:348] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } } peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } }
I20260806 19:39:18.179399 26946 raft_consensus.cc:374] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:39:18.179510 26946 raft_consensus.cc:729] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc359d524bc4759845f6781e9f6738d, State: Initialized, Role: FOLLOWER
I20260806 19:39:18.180011 26944 raft_consensus.cc:348] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } } peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } }
I20260806 19:39:18.180475 26944 raft_consensus.cc:374] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:39:18.180275 26946 consensus_queue.cc:260] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } } peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } }
I20260806 19:39:18.180585 26944 raft_consensus.cc:729] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26a908fd090745318dda758213a0acb1, State: Initialized, Role: FOLLOWER
I20260806 19:39:18.181258 26944 consensus_queue.cc:260] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } } peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } }
I20260806 19:39:18.182113 26789 heartbeater.cc:503] Master 127.10.25.62:43233 was elected leader, sending a full tablet report...
I20260806 19:39:18.182672 26925 heartbeater.cc:503] Master 127.10.25.62:43233 was elected leader, sending a full tablet report...
I20260806 19:39:18.182893 26946 ts_tablet_manager.cc:1428] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260806 19:39:18.183218 26944 ts_tablet_manager.cc:1428] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260806 19:39:18.183404 26945 raft_consensus.cc:348] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } } peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } }
I20260806 19:39:18.183931 26945 raft_consensus.cc:374] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:39:18.184055 26945 raft_consensus.cc:729] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9da9351acb7e41539f7b2f0ff4da3aa7, State: Initialized, Role: FOLLOWER
I20260806 19:39:18.184850 26945 consensus_queue.cc:260] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } } peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } }
I20260806 19:39:18.186935 26945 ts_tablet_manager.cc:1428] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7: Time spent starting tablet: real 0.018s user 0.015s sys 0.001s
W20260806 19:39:18.202906 26656 tablet.cc:2369] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:39:18.214730 26951 raft_consensus.cc:482] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:39:18.215116 26951 raft_consensus.cc:504] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } } peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } }
I20260806 19:39:18.218119 26951 leader_election.cc:288] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9da9351acb7e41539f7b2f0ff4da3aa7 (127.10.25.1:45971), 5dc359d524bc4759845f6781e9f6738d (127.10.25.2:41159)
I20260806 19:39:18.226940 26939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.218723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:57283 (local address 127.10.25.1:45971)
0806 19:39:18.218948 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:18.218954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:18.218977 (+ 23us) server_negotiation.cc:407] Connection header received
0806 19:39:18.219058 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:18.219065 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:18.219157 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:18.219269 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:39:18.219971 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.221142 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.224396 (+ 3254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.225879 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.226061 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.226208 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:18.226584 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 19:39:18.226724 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 19:39:18.227186 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.218436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:45971 (local address 127.10.25.3:57283)
0806 19:39:18.218719 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:18.218740 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:18.218903 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:18.219478 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:18.219483 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:18.219518 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:39:18.219776 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:18.219788 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.221311 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.221319 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:18.224163 (+ 2844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:18.224184 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.226109 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.226115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:18.226308 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.226341 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 19:39:18.226551 (+ 210us) client_negotiation.cc:240] Negotiation successful
0806 19:39:18.226752 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":62}
I20260806 19:39:18.227720 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.219164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:54195 (local address 127.10.25.2:41159)
0806 19:39:18.219440 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:18.219446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:18.219771 (+ 325us) server_negotiation.cc:407] Connection header received
0806 19:39:18.219909 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:18.219914 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:18.219989 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:18.220067 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:39:18.220876 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.222186 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.224865 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.226321 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.226447 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.226577 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:18.227211 (+ 634us) server_negotiation.cc:299] Negotiation successful
0806 19:39:18.227434 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 19:39:18.228116 26953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.218979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:41159 (local address 127.10.25.3:54195)
0806 19:39:18.219643 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:18.219664 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:18.219779 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:18.220127 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:18.220132 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:18.220162 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:39:18.220598 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:18.220613 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.222394 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.222398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:18.224649 (+ 2251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:18.224664 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.226645 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.226653 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:18.226859 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.226901 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 19:39:18.227030 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 19:39:18.227583 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":1,"thread_start_us":459,"threads_started":1}
I20260806 19:39:18.228639 26608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1eecca4bf1249768a2fb332d0d7f807" candidate_uuid: "26a908fd090745318dda758213a0acb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" is_pre_election: true
I20260806 19:39:18.229219 26608 raft_consensus.cc:2454] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26a908fd090745318dda758213a0acb1 in term 0.
I20260806 19:39:18.229276 26744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1eecca4bf1249768a2fb332d0d7f807" candidate_uuid: "26a908fd090745318dda758213a0acb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc359d524bc4759845f6781e9f6738d" is_pre_election: true
I20260806 19:39:18.230003 26744 raft_consensus.cc:2454] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26a908fd090745318dda758213a0acb1 in term 0.
I20260806 19:39:18.230150 26811 leader_election.cc:302] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26a908fd090745318dda758213a0acb1, 9da9351acb7e41539f7b2f0ff4da3aa7; no voters:
I20260806 19:39:18.230696 26951 raft_consensus.cc:2781] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:39:18.230873 26951 raft_consensus.cc:482] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:39:18.230996 26951 raft_consensus.cc:3037] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:39:18.233747 26951 raft_consensus.cc:504] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } } peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } }
I20260806 19:39:18.234689 26951 leader_election.cc:288] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [CANDIDATE]: Term 1 election: Requested vote from peers 9da9351acb7e41539f7b2f0ff4da3aa7 (127.10.25.1:45971), 5dc359d524bc4759845f6781e9f6738d (127.10.25.2:41159)
I20260806 19:39:18.235811 26744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1eecca4bf1249768a2fb332d0d7f807" candidate_uuid: "26a908fd090745318dda758213a0acb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc359d524bc4759845f6781e9f6738d"
I20260806 19:39:18.235813 26608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1eecca4bf1249768a2fb332d0d7f807" candidate_uuid: "26a908fd090745318dda758213a0acb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7"
I20260806 19:39:18.236161 26608 raft_consensus.cc:3037] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:39:18.236161 26744 raft_consensus.cc:3037] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:39:18.240178 26608 raft_consensus.cc:2454] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26a908fd090745318dda758213a0acb1 in term 1.
I20260806 19:39:18.240180 26744 raft_consensus.cc:2454] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26a908fd090745318dda758213a0acb1 in term 1.
I20260806 19:39:18.240871 26811 leader_election.cc:302] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26a908fd090745318dda758213a0acb1, 5dc359d524bc4759845f6781e9f6738d; no voters:
I20260806 19:39:18.241344 26951 raft_consensus.cc:2781] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:39:18.241971 26951 raft_consensus.cc:686] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [term 1 LEADER]: Becoming Leader. State: Replica: 26a908fd090745318dda758213a0acb1, State: Running, Role: LEADER
I20260806 19:39:18.242693 26951 consensus_queue.cc:237] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } } peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } }
I20260806 19:39:18.248898 26462 catalog_manager.cc:5495] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26a908fd090745318dda758213a0acb1 (127.10.25.3). New cstate: current_term: 1 leader_uuid: "26a908fd090745318dda758213a0acb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } health_report { overall_health: HEALTHY } } }
W20260806 19:39:18.274600 26926 tablet.cc:2369] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:39:18.297181 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.289762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34286 (local address 127.10.25.3:36477)
0806 19:39:18.289988 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:18.289993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:18.290021 (+ 28us) server_negotiation.cc:407] Connection header received
0806 19:39:18.290321 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:18.290327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:18.290443 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:18.290599 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:39:18.291063 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.292115 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.293988 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.294534 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.294662 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.295464 (+ 802us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:39:18.296243 (+ 779us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:39:18.296637 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:18.296698 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 19:39:18.296791 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 19:39:18.306854 26608 raft_consensus.cc:1261] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 1 FOLLOWER]: Refusing update from remote peer 26a908fd090745318dda758213a0acb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:39:18.306854 26744 raft_consensus.cc:1261] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d [term 1 FOLLOWER]: Refusing update from remote peer 26a908fd090745318dda758213a0acb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:39:18.308065 26954 consensus_queue.cc:1035] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:39:18.308681 26951 consensus_queue.cc:1035] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:39:18.328945 26959 mvcc.cc:205] Tried to move back new op lower bound from 7315640968412692480 to 7315640968167591936. Current Snapshot: MvccSnapshot[applied={T|T < 7315640968412692480}]
I20260806 19:39:18.345249 26958 mvcc.cc:205] Tried to move back new op lower bound from 7315640968412692480 to 7315640968167591936. Current Snapshot: MvccSnapshot[applied={T|T < 7315640968412692480}]
I20260806 19:39:18.349545 26462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:39:18.349941 26462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:39:18.354645 26462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Sending DeleteTablet for 3 replicas of tablet d1eecca4bf1249768a2fb332d0d7f807
I20260806 19:39:18.356937 26724 tablet_service.cc:1514] Processing DeleteTablet for tablet d1eecca4bf1249768a2fb332d0d7f807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:39:18 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:52576
I20260806 19:39:18.357437 26588 tablet_service.cc:1514] Processing DeleteTablet for tablet d1eecca4bf1249768a2fb332d0d7f807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:39:18 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:58582
I20260806 19:39:18.358140 26968 tablet_replica.cc:330] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d: stopping tablet replica
I20260806 19:39:18.358479 26969 tablet_replica.cc:330] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7: stopping tablet replica
I20260806 19:39:18.358654 26968 raft_consensus.cc:2229] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:39:18.358879 26969 raft_consensus.cc:2229] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:39:18.359266 26968 raft_consensus.cc:2258] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:39:18.359237 26860 tablet_service.cc:1514] Processing DeleteTablet for tablet d1eecca4bf1249768a2fb332d0d7f807 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:39:18 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:34270
I20260806 19:39:18.359367 26969 raft_consensus.cc:2258] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:39:18.360247 26970 tablet_replica.cc:330] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1: stopping tablet replica
I20260806 19:39:18.360674 26970 raft_consensus.cc:2229] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:39:18.360931 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 26525
I20260806 19:39:18.361406 26970 raft_consensus.cc:2258] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:39:18.362254 26969 ts_tablet_manager.cc:1905] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:39:18.364065 26970 ts_tablet_manager.cc:1905] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:39:18.373260 26970 ts_tablet_manager.cc:1918] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:39:18.373550 26970 log.cc:1196] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/d1eecca4bf1249768a2fb332d0d7f807
I20260806 19:39:18.374182 26970 ts_tablet_manager.cc:1939] T d1eecca4bf1249768a2fb332d0d7f807 P 26a908fd090745318dda758213a0acb1: Deleting consensus metadata
I20260806 19:39:18.376318 26449 catalog_manager.cc:4841] TS 26a908fd090745318dda758213a0acb1 (127.10.25.3:36477): tablet d1eecca4bf1249768a2fb332d0d7f807 (table test-table [id=fefd577442564b1fb74f5ad3cc8e9223]) successfully deleted
I20260806 19:39:18.379858 26969 ts_tablet_manager.cc:1918] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:39:18.380091 26969 log.cc:1196] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/d1eecca4bf1249768a2fb332d0d7f807
I20260806 19:39:18.376402 26968 ts_tablet_manager.cc:1905] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:39:18.380729 26969 ts_tablet_manager.cc:1939] T d1eecca4bf1249768a2fb332d0d7f807 P 9da9351acb7e41539f7b2f0ff4da3aa7: Deleting consensus metadata
I20260806 19:39:18.382943 26447 catalog_manager.cc:4841] TS 9da9351acb7e41539f7b2f0ff4da3aa7 (127.10.25.1:45971): tablet d1eecca4bf1249768a2fb332d0d7f807 (table test-table [id=fefd577442564b1fb74f5ad3cc8e9223]) successfully deleted
Aug 06 19:39:18 dist-test-slave-ss4n krb5kdc[26417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:39:18.397311 26939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.362379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58584 (local address 127.10.25.1:45971)
0806 19:39:18.362588 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:18.362595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:18.372353 (+ 9758us) server_negotiation.cc:407] Connection header received
0806 19:39:18.372730 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:18.372736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:18.372838 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:18.372919 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:18.375536 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.377927 (+ 2391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.380351 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.381094 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.381211 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.387692 (+ 6481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:18.387717 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:18.387739 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:18.387797 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:18.392232 (+ 4435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:18.393427 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:18.393439 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:18.393446 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:18.393688 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:18.394237 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:18.394244 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:18.394248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:18.394789 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:18.396416 (+ 1627us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:18.396615 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 19:39:18.396893 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 19:39:18.398581 26650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:39:18.402725 26968 ts_tablet_manager.cc:1918] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:39:18.403046 26968 log.cc:1196] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/d1eecca4bf1249768a2fb332d0d7f807
I20260806 19:39:18.403741 26968 ts_tablet_manager.cc:1939] T d1eecca4bf1249768a2fb332d0d7f807 P 5dc359d524bc4759845f6781e9f6738d: Deleting consensus metadata
I20260806 19:39:18.406111 26447 catalog_manager.cc:4841] TS 5dc359d524bc4759845f6781e9f6738d (127.10.25.2:41159): tablet d1eecca4bf1249768a2fb332d0d7f807 (table test-table [id=fefd577442564b1fb74f5ad3cc8e9223]) successfully deleted
I20260806 19:39:18.507220 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 26525
I20260806 19:39:18.538115 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 26661
Aug 06 19:39:18 dist-test-slave-ss4n krb5kdc[26417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:39:18.563742 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.539270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52578 (local address 127.10.25.2:41159)
0806 19:39:18.539474 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:18.539479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:18.539502 (+ 23us) server_negotiation.cc:407] Connection header received
0806 19:39:18.539796 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:18.539802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:18.539938 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:18.540085 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:18.542906 (+ 2821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.543773 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.547258 (+ 3485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.547789 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.547939 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.554948 (+ 7009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:18.554972 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:18.554990 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:18.555050 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:18.558938 (+ 3888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:18.560052 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:18.560062 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:18.560064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:18.560242 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:18.560756 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:18.560760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:18.560763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:18.561231 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:18.561473 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:18.562865 (+ 1392us) server_negotiation.cc:299] Negotiation successful
0806 19:39:18.563389 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 19:39:18.564625 26784 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:39:18.663300 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 26661
I20260806 19:39:18.686949 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 26795
Aug 06 19:39:18 dist-test-slave-ss4n krb5kdc[26417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:39:18.715195 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:18.692228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34300 (local address 127.10.25.3:36477)
0806 19:39:18.692485 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:18.692491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:18.692519 (+ 28us) server_negotiation.cc:407] Connection header received
0806 19:39:18.692663 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:18.692674 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:18.692778 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:18.692905 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:18.694552 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.695761 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.697412 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:18.698215 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:18.698416 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:18.703957 (+ 5541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:18.703993 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:18.704039 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:18.704105 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:18.710841 (+ 6736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:18.711889 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:18.711900 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:18.711903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:18.712134 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:18.712621 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:18.712626 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:18.712644 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:18.713204 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:18.713585 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:18.714323 (+ 738us) server_negotiation.cc:299] Negotiation successful
0806 19:39:18.714751 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 19:39:18.716176 26920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:39:18.851826 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 26795
I20260806 19:39:18.876760 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 26430
I20260806 19:39:18.878187 26494 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:39:18.997089 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 26430
I20260806 19:39:39.022639 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:43233
--webserver_interface=127.10.25.62
--webserver_port=36495
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:43233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:39:39.401816 26976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:39:39.402249 26976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:39:39.402416 26976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:39:39.410337 26976 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 19:39:39.410506 26976 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 19:39:39.410615 26976 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 19:39:39.410706 26976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:39:39.410807 26976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:39:39.410934 26976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:39:39.411031 26976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:39:39.411113 26976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:39:39.422431 26976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:43233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:43233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=36495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:39:39.423943 26976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:39:39.425441 26976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:39:39.452732 26985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:39:39.452620 26982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:39:39.452675 26983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:39:39.453867 26976 server_base.cc:1034] running on GCE node
I20260806 19:39:39.454990 26976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:39:39.456988 26976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:39:39.458338 26976 hybrid_clock.cc:648] HybridClock initialized: now 1786045179458265 us; error 82 us; skew 500 ppm
I20260806 19:39:39.458840 26976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:39:39 dist-test-slave-ss4n krb5kdc[26417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045179, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:39:39.467211 26976 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:39:39.469664 26976 webserver.cc:460] Webserver started at http://127.10.25.62:36495/ using document root <none> and password file <none>
I20260806 19:39:39.470659 26976 fs_manager.cc:359] Metadata directory not provided
I20260806 19:39:39.470805 26976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:39:39.478665 26976 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 19:39:39.482473 26992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:39.484898 26976 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:39:39.485122 26976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "9d61ccd09113402aa039f1960b2dc3a6"
format_stamp: "Formatted at 2026-08-06 19:39:15 on dist-test-slave-ss4n"
server_key: "db23c1de69d2ef087708a7a5d487d557"
server_key_iv: "042c540f263c0edf0c53049dc0757b21"
server_key_version: "encryptionkey@0"
I20260806 19:39:39.486632 26976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:39:39.509243 26976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:39:39.514842 26976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:39:39.515175 26976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:39:39.540464 26976 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:43233
I20260806 19:39:39.540501 27043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:43233 every 8 connection(s)
I20260806 19:39:39.542552 26976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:39:39.548130 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 26976
I20260806 19:39:39.549086 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:45971
--local_ip_for_outbound_sockets=127.10.25.1
--tserver_master_addrs=127.10.25.62:43233
--webserver_port=36833
--webserver_interface=127.10.25.1
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:39:39.550796 27044 sys_catalog.cc:263] Verifying existing consensus state
I20260806 19:39:39.555199 27044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Bootstrap starting.
I20260806 19:39:39.577347 27044 log.cc:824] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Log is configured to *not* fsync() on all Append() calls
I20260806 19:39:39.591109 27044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:39:39.591755 27044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Bootstrap complete.
I20260806 19:39:39.601737 27044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } }
I20260806 19:39:39.602790 27044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d61ccd09113402aa039f1960b2dc3a6, State: Initialized, Role: FOLLOWER
I20260806 19:39:39.603479 27044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } }
I20260806 19:39:39.603765 27044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:39:39.603904 27044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:39:39.604090 27044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 1 FOLLOWER]: Advancing to term 2
I20260806 19:39:39.608474 27044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } }
I20260806 19:39:39.609103 27044 leader_election.cc:302] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d61ccd09113402aa039f1960b2dc3a6; no voters:
I20260806 19:39:39.609910 27044 leader_election.cc:288] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 19:39:39.610040 27048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 19:39:39.611564 27048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 2 LEADER]: Becoming Leader. State: Replica: 9d61ccd09113402aa039f1960b2dc3a6, State: Running, Role: LEADER
I20260806 19:39:39.612227 27048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } }
I20260806 19:39:39.612658 27044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:39:39.615260 27050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d61ccd09113402aa039f1960b2dc3a6. Latest consensus state: current_term: 2 leader_uuid: "9d61ccd09113402aa039f1960b2dc3a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } } }
I20260806 19:39:39.615825 27050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [sys.catalog]: This master's current role is: LEADER
I20260806 19:39:39.615813 27049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9d61ccd09113402aa039f1960b2dc3a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } } }
I20260806 19:39:39.616195 27049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [sys.catalog]: This master's current role is: LEADER
I20260806 19:39:39.625752 27053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:39:39.641074 27053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:39:39.643258 27053 catalog_manager.cc:1259] Loaded cluster ID: eb8501212f7b48e4a6bfe0d15f50defe
I20260806 19:39:39.643414 27053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:39:39.649336 27053 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:39:39.652207 27053 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Loaded TSK: 0
I20260806 19:39:39.706310 27053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Generated new TSK 1
I20260806 19:39:39.706751 27053 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 19:39:39.988761 27046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:39:39.989156 27046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:39:39.989341 27046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:39:39.998296 27046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:39:39.998620 27046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:39:39.998852 27046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:39:40.011536 27046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:45971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=36833
--tserver_master_addrs=127.10.25.62:43233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:39:40.013118 27046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:39:40.014743 27046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:39:40.030260 27072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:39:40.030342 27071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:39:40.030588 27074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:39:40.032644 27046 server_base.cc:1034] running on GCE node
I20260806 19:39:40.033624 27046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:39:40.035277 27046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:39:40.036628 27046 hybrid_clock.cc:648] HybridClock initialized: now 1786045180036550 us; error 57 us; skew 500 ppm
I20260806 19:39:40.037407 27046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:39:40 dist-test-slave-ss4n krb5kdc[26417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:39:40.041908 27046 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:39:40.043892 27046 webserver.cc:460] Webserver started at http://127.10.25.1:36833/ using document root <none> and password file <none>
I20260806 19:39:40.044713 27046 fs_manager.cc:359] Metadata directory not provided
I20260806 19:39:40.044898 27046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:39:40.051414 27046 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 19:39:40.054801 27082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:40.056529 27046 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 19:39:40.056740 27046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "9da9351acb7e41539f7b2f0ff4da3aa7"
format_stamp: "Formatted at 2026-08-06 19:39:16 on dist-test-slave-ss4n"
server_key: "8b85c43e172ff9ac7c1a0c8c07193e2c"
server_key_iv: "fee25853912f5dbc3ed52b790e402fd8"
server_key_version: "encryptionkey@0"
I20260806 19:39:40.057912 27046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:39:40.082085 27046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:39:40.087160 27046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:39:40.087498 27046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:39:40.088735 27046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:39:40.090399 27046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:39:40.090519 27046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:40.090679 27046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:39:40.090780 27046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:40.132541 27046 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:45971
I20260806 19:39:40.132705 27195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:45971 every 8 connection(s)
I20260806 19:39:40.134433 27046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:39:40.144837 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 27046
I20260806 19:39:40.145849 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:41159
--local_ip_for_outbound_sockets=127.10.25.2
--tserver_master_addrs=127.10.25.62:43233
--webserver_port=46557
--webserver_interface=127.10.25.2
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:39:40 dist-test-slave-ss4n krb5kdc[26417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:39:40.166867 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:40.137930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:55063 (local address 127.10.25.62:43233)
0806 19:39:40.138551 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:40.138567 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:40.139347 (+ 780us) server_negotiation.cc:407] Connection header received
0806 19:39:40.141910 (+ 2563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:40.141924 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:40.142397 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:40.142780 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:40.144572 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:40.146974 (+ 2402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:40.148981 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:40.150342 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:40.150613 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:40.156739 (+ 6126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:40.156757 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:40.156787 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:40.156868 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:40.160712 (+ 3844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:40.162069 (+ 1357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:40.162078 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:40.162084 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:40.162380 (+ 296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:40.163161 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:40.163165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:40.163167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:40.163853 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:40.164017 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:40.165173 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0806 19:39:40.165789 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":299,"threads_started":1}
I20260806 19:39:40.167167 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:40.138294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:55063)
0806 19:39:40.139142 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:40.139185 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:40.141144 (+ 1959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:40.142966 (+ 1822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:40.142975 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:40.143638 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:39:40.144290 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:40.144312 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:40.147245 (+ 2933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:40.147250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:40.148761 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:40.148780 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:40.150853 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:40.150862 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:40.151388 (+ 526us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:40.152414 (+ 1026us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:39:40.152456 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:39:40.156244 (+ 3788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:39:40.160972 (+ 4728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:39:40.160986 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:39:40.161008 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:39:40.161894 (+ 886us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:39:40.162604 (+ 710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:39:40.162612 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:39:40.162619 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:39:40.162962 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:39:40.164049 (+ 1087us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:39:40.164062 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:39:40.164567 (+ 505us) client_negotiation.cc:769] Sending connection context
0806 19:39:40.164910 (+ 343us) client_negotiation.cc:240] Negotiation successful
0806 19:39:40.165622 (+ 712us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":333,"threads_started":1}
I20260806 19:39:40.170674 27196 heartbeater.cc:347] Connected to a master server at 127.10.25.62:43233
I20260806 19:39:40.171065 27196 heartbeater.cc:464] Registering TS with master...
I20260806 19:39:40.172073 27196 heartbeater.cc:511] Master 127.10.25.62:43233 requested a full tablet report, sending...
I20260806 19:39:40.175421 27009 ts_manager.cc:194] Registered new tserver with Master: 9da9351acb7e41539f7b2f0ff4da3aa7 (127.10.25.1:45971)
I20260806 19:39:40.178042 27009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:55063
W20260806 19:39:40.546846 27200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:39:40.547293 27200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:39:40.547523 27200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:39:40.557402 27200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:39:40.557657 27200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:39:40.557869 27200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:39:40.571475 27200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:41159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=46557
--tserver_master_addrs=127.10.25.62:43233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:39:40.573241 27200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:39:40.574934 27200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:39:40.599116 27205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:39:40.599349 27206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:39:40.599133 27208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:39:40.600338 27200 server_base.cc:1034] running on GCE node
I20260806 19:39:40.601234 27200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:39:40.603004 27200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:39:40.604332 27200 hybrid_clock.cc:648] HybridClock initialized: now 1786045180604241 us; error 83 us; skew 500 ppm
I20260806 19:39:40.604842 27200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:39:40 dist-test-slave-ss4n krb5kdc[26417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:39:40.609179 27200 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:39:40.611447 27200 webserver.cc:460] Webserver started at http://127.10.25.2:46557/ using document root <none> and password file <none>
I20260806 19:39:40.612370 27200 fs_manager.cc:359] Metadata directory not provided
I20260806 19:39:40.612572 27200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:39:40.620028 27200 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:39:40.623591 27216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:40.625437 27200 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 19:39:40.625787 27200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "5dc359d524bc4759845f6781e9f6738d"
format_stamp: "Formatted at 2026-08-06 19:39:17 on dist-test-slave-ss4n"
server_key: "45cffeac6f03a84ff047ad823c0b0e6d"
server_key_iv: "b855018a4064c41afe4f9f2ad64d95e8"
server_key_version: "encryptionkey@0"
I20260806 19:39:40.626879 27200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:39:40.652626 27200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:39:40.658779 27200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:39:40.659169 27200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:39:40.660447 27200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:39:40.662786 27200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:39:40.662951 27200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:40.663093 27200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:39:40.663179 27200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:40.713377 27200 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:41159
I20260806 19:39:40.713455 27329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:41159 every 8 connection(s)
I20260806 19:39:40.715171 27200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:39:40.715919 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 27200
I20260806 19:39:40.716876 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:36477
--local_ip_for_outbound_sockets=127.10.25.3
--tserver_master_addrs=127.10.25.62:43233
--webserver_port=40743
--webserver_interface=127.10.25.3
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 19:39:40 dist-test-slave-ss4n krb5kdc[26417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:39:40.742115 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:40.719415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:43677 (local address 127.10.25.62:43233)
0806 19:39:40.720022 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:40.720027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:40.721306 (+ 1279us) server_negotiation.cc:407] Connection header received
0806 19:39:40.723471 (+ 2165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:40.723479 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:40.723597 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:40.723733 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:40.725671 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:40.726421 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:40.728800 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:40.729381 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:40.729582 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:40.734502 (+ 4920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:40.734509 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:40.734512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:40.734558 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:40.737959 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:40.739047 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:40.739052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:40.739055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:40.739232 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:40.739861 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:40.739865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:40.739868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:40.740224 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:40.740437 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:40.741358 (+ 921us) server_negotiation.cc:299] Negotiation successful
0806 19:39:40.741703 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":349,"threads_started":1}
I20260806 19:39:40.743419 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:40.720068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:43677)
0806 19:39:40.721092 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:40.721138 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:40.723100 (+ 1962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:40.724041 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:40.724051 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:40.724697 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:39:40.725375 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:40.725398 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:40.726655 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:40.726660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:40.728573 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:40.728593 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:40.729701 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:40.729712 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:40.730002 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:40.730880 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:39:40.730921 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:39:40.734146 (+ 3225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:39:40.738209 (+ 4063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:39:40.738223 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:39:40.738254 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:39:40.738854 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:39:40.739380 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:39:40.739386 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:39:40.739390 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:39:40.739692 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:39:40.740404 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:39:40.740414 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:39:40.740858 (+ 444us) client_negotiation.cc:769] Sending connection context
0806 19:39:40.741192 (+ 334us) client_negotiation.cc:240] Negotiation successful
0806 19:39:40.741877 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":451,"threads_started":1}
I20260806 19:39:40.745471 27330 heartbeater.cc:347] Connected to a master server at 127.10.25.62:43233
I20260806 19:39:40.746052 27330 heartbeater.cc:464] Registering TS with master...
I20260806 19:39:40.747074 27330 heartbeater.cc:511] Master 127.10.25.62:43233 requested a full tablet report, sending...
I20260806 19:39:40.749786 27009 ts_manager.cc:194] Registered new tserver with Master: 5dc359d524bc4759845f6781e9f6738d (127.10.25.2:41159)
I20260806 19:39:40.751065 27009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:43677
W20260806 19:39:41.132288 27334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:39:41.132810 27334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:39:41.132982 27334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:39:41.144555 27334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:39:41.144721 27334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:39:41.144850 27334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:39:41.156261 27334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:36477
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=40743
--tserver_master_addrs=127.10.25.62:43233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:39:41.157758 27334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:39:41.159219 27334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:39:41.175423 27339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:39:41.175341 27342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:39:41.175386 27340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:39:41.176437 27334 server_base.cc:1034] running on GCE node
I20260806 19:39:41.177181 27334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:39:41.178582 27334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:39:41.179845 27334 hybrid_clock.cc:648] HybridClock initialized: now 1786045181179800 us; error 53 us; skew 500 ppm
I20260806 19:39:41.180285 27334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:39:41.183254 27196 heartbeater.cc:503] Master 127.10.25.62:43233 was elected leader, sending a full tablet report...
Aug 06 19:39:41 dist-test-slave-ss4n krb5kdc[26417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045181, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:39:41.184422 27334 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:39:41.186571 27334 webserver.cc:460] Webserver started at http://127.10.25.3:40743/ using document root <none> and password file <none>
I20260806 19:39:41.187379 27334 fs_manager.cc:359] Metadata directory not provided
I20260806 19:39:41.187520 27334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:39:41.194680 27334 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 19:39:41.198740 27349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:41.200564 27334 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260806 19:39:41.200824 27334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "26a908fd090745318dda758213a0acb1"
format_stamp: "Formatted at 2026-08-06 19:39:17 on dist-test-slave-ss4n"
server_key: "dfb314d0f2acc657b313e0e6ce440d49"
server_key_iv: "f485b6ad44a4bd58ce2643ab73c0dba9"
server_key_version: "encryptionkey@0"
I20260806 19:39:41.202229 27334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:39:41.221662 27334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:39:41.227744 27334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:39:41.228164 27334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:39:41.229647 27334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:39:41.231573 27334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:39:41.231690 27334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:41.231842 27334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:39:41.231925 27334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:39:41.273433 27334 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:36477
I20260806 19:39:41.273499 27463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:36477 every 8 connection(s)
I20260806 19:39:41.275336 27334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:39:41.276187 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 27334
Aug 06 19:39:41 dist-test-slave-ss4n krb5kdc[26417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045181, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:39:41.297552 27467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.277595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42348 (local address 127.10.25.62:43233)
0806 19:39:41.279642 (+ 2047us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:41.279653 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:41.279688 (+ 35us) server_negotiation.cc:407] Connection header received
0806 19:39:41.279793 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:41.279803 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:41.279955 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:41.280146 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:41.282920 (+ 2774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.283783 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.285420 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.285991 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.286160 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.289271 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:41.289277 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:41.289281 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:41.289323 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:41.293595 (+ 4272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:41.294778 (+ 1183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:41.294785 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:41.294788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:41.294990 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:41.295595 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:41.295602 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:41.295608 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:41.296000 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:41.296351 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:41.296767 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 19:39:41.297075 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1858,"thread_start_us":1690,"threads_started":1}
I20260806 19:39:41.303227 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.279121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:44423 (local address 127.10.25.62:43233)
0806 19:39:41.282333 (+ 3212us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:41.282340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:41.282368 (+ 28us) server_negotiation.cc:407] Connection header received
0806 19:39:41.284078 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:41.284084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:41.284219 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:41.284367 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:41.286426 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.287399 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.289361 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.290087 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.290272 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.295615 (+ 5343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:41.295622 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:41.295625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:41.295660 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:41.299006 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:41.300162 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:41.300171 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:41.300177 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:41.300359 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:41.301105 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:41.301110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:41.301113 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:41.301582 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:41.301756 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:41.302644 (+ 888us) server_negotiation.cc:299] Negotiation successful
0806 19:39:41.302892 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3046,"thread_start_us":2826,"threads_started":1}
I20260806 19:39:41.304291 27471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.279782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:44423)
0806 19:39:41.281338 (+ 1556us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:41.281432 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:41.283722 (+ 2290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:41.284696 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:41.284707 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:41.285382 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:39:41.286191 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:41.286213 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.287636 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.287642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:41.289159 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:41.289176 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.290403 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.290426 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:41.290760 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.291811 (+ 1051us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:39:41.291882 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:39:41.295335 (+ 3453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:39:41.299224 (+ 3889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:39:41.299234 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:39:41.299286 (+ 52us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:39:41.299998 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:39:41.300597 (+ 599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:39:41.300603 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:39:41.300606 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:39:41.300881 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:39:41.301753 (+ 872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:39:41.301760 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:39:41.302188 (+ 428us) client_negotiation.cc:769] Sending connection context
0806 19:39:41.302459 (+ 271us) client_negotiation.cc:240] Negotiation successful
0806 19:39:41.302906 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"thread_start_us":801,"threads_started":1}
I20260806 19:39:41.306152 27464 heartbeater.cc:347] Connected to a master server at 127.10.25.62:43233
I20260806 19:39:41.306535 27464 heartbeater.cc:464] Registering TS with master...
I20260806 19:39:41.307516 27464 heartbeater.cc:511] Master 127.10.25.62:43233 requested a full tablet report, sending...
I20260806 19:39:41.309633 27009 ts_manager.cc:194] Registered new tserver with Master: 26a908fd090745318dda758213a0acb1 (127.10.25.3:36477)
I20260806 19:39:41.311416 27009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:44423
I20260806 19:39:41.314234 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:39:41.322682 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.315304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42356 (local address 127.10.25.62:43233)
0806 19:39:41.315518 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:41.315527 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:41.316044 (+ 517us) server_negotiation.cc:407] Connection header received
0806 19:39:41.316365 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:41.316370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:41.316512 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:41.316689 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:39:41.317199 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.318092 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.320018 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.320531 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.320722 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.321364 (+ 642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:39:41.321770 (+ 406us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 19:39:41.322293 (+ 523us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42356: token expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 19:39:41.322854 27470 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42356: token expired
W20260806 19:39:41.323518 26931 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.25.62:43233)
I20260806 19:39:41.340986 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.324250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42370 (local address 127.10.25.62:43233)
0806 19:39:41.324507 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:41.324514 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:41.324538 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:39:41.324758 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:41.324763 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:41.324840 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:41.324968 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:41.326873 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.327735 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.329668 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.330182 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.330456 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.334318 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:41.334324 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:41.334328 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:41.334375 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:41.337892 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:41.338770 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:41.338775 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:41.338777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:41.338952 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:41.339374 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:41.339378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:41.339380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:41.339727 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:41.340015 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:41.340370 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 19:39:41.340635 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 19:39:41.345136 27009 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:39:41.354028 27009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:39:41.403254 27482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.389332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55392 (local address 127.10.25.1:45971)
0806 19:39:41.390803 (+ 1471us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:41.390817 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:41.391671 (+ 854us) server_negotiation.cc:407] Connection header received
0806 19:39:41.392012 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:41.392022 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:41.392275 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:41.392538 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:39:41.393448 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.395644 (+ 2196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.399428 (+ 3784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.401034 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.401146 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.401406 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:41.402108 (+ 702us) server_negotiation.cc:299] Negotiation successful
0806 19:39:41.402582 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1239,"thread_start_us":1050,"threads_started":1}
I20260806 19:39:41.403379 27477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.388886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:45971 (local address 127.0.0.1:55392)
0806 19:39:41.391486 (+ 2600us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:41.391510 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:41.391696 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:41.392529 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:41.392534 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:41.392566 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:39:41.393159 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:41.393177 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.395818 (+ 2641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.395824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:41.398692 (+ 2868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:41.398716 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.401642 (+ 2926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.401648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:41.401827 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.401870 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 19:39:41.402043 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 19:39:41.402567 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2415,"mutex_wait_us":1,"thread_start_us":725,"threads_started":1}
I20260806 19:39:41.404210 27481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.376493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:36477 (local address 127.0.0.1:36640)
0806 19:39:41.389627 (+ 13134us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:41.389715 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:41.389982 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:41.393970 (+ 3988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:41.393978 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:41.394019 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:39:41.394534 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:41.394551 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.397641 (+ 3090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.397648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:41.400198 (+ 2550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:41.400216 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.403047 (+ 2831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.403058 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:41.403333 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.403386 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 19:39:41.403526 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 19:39:41.403734 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12930,"thread_start_us":10173,"threads_started":1}
I20260806 19:39:41.405400 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.377469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36640 (local address 127.10.25.3:36477)
0806 19:39:41.392893 (+ 15424us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:41.392902 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:41.392939 (+ 37us) server_negotiation.cc:407] Connection header received
0806 19:39:41.393039 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:41.393048 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:41.393286 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:41.393793 (+ 507us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:39:41.394841 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.397248 (+ 2407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.400565 (+ 3317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.402395 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.403070 (+ 675us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.403335 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:41.404030 (+ 695us) server_negotiation.cc:299] Negotiation successful
0806 19:39:41.404942 (+ 912us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15188,"thread_start_us":15941,"threads_started":1}
I20260806 19:39:41.405683 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.375660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:41159 (local address 127.0.0.1:35884)
0806 19:39:41.389627 (+ 13967us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:41.390846 (+ 1219us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:41.391007 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:41.392171 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:41.392179 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:41.392215 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:39:41.393114 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:41.393156 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.396505 (+ 3349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.396515 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:41.399634 (+ 3119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:41.399664 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.404488 (+ 4824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.404501 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:41.404756 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.404799 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 19:39:41.404932 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 19:39:41.405105 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13550,"mutex_wait_us":1,"thread_start_us":12544,"threads_started":1}
I20260806 19:39:41.406656 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.375894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35884 (local address 127.10.25.2:41159)
0806 19:39:41.388010 (+ 12116us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:41.388144 (+ 134us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:41.390986 (+ 2842us) server_negotiation.cc:407] Connection header received
0806 19:39:41.391188 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:41.391201 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:41.391476 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:41.391743 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:39:41.393546 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.395881 (+ 2335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.399947 (+ 4066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.402069 (+ 2122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.402262 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.402556 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:41.406024 (+ 3468us) server_negotiation.cc:299] Negotiation successful
0806 19:39:41.406275 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11916,"thread_start_us":11792,"threads_started":1}
I20260806 19:39:41.408273 27130 tablet_service.cc:1467] Processing CreateTablet for tablet 58e5f292c5b44445bdd7e357bed97121 (DEFAULT_TABLE table=test-table [id=267f00aa7a5446d69c053c9914637764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:39:41.409888 27130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58e5f292c5b44445bdd7e357bed97121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:39:41.409979 27398 tablet_service.cc:1467] Processing CreateTablet for tablet 58e5f292c5b44445bdd7e357bed97121 (DEFAULT_TABLE table=test-table [id=267f00aa7a5446d69c053c9914637764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:39:41.411751 27398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58e5f292c5b44445bdd7e357bed97121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:39:41.413245 27264 tablet_service.cc:1467] Processing CreateTablet for tablet 58e5f292c5b44445bdd7e357bed97121 (DEFAULT_TABLE table=test-table [id=267f00aa7a5446d69c053c9914637764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:39:41.414808 27264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58e5f292c5b44445bdd7e357bed97121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:39:41.424783 27483 tablet_bootstrap.cc:492] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7: Bootstrap starting.
I20260806 19:39:41.428733 27483 tablet_bootstrap.cc:654] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7: Neither blocks nor log segments found. Creating new log.
I20260806 19:39:41.430487 27484 tablet_bootstrap.cc:492] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1: Bootstrap starting.
I20260806 19:39:41.430356 27483 log.cc:824] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7: Log is configured to *not* fsync() on all Append() calls
I20260806 19:39:41.433641 27486 tablet_bootstrap.cc:492] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d: Bootstrap starting.
I20260806 19:39:41.436466 27484 tablet_bootstrap.cc:654] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1: Neither blocks nor log segments found. Creating new log.
I20260806 19:39:41.437124 27486 tablet_bootstrap.cc:654] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d: Neither blocks nor log segments found. Creating new log.
I20260806 19:39:41.438364 27486 log.cc:824] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d: Log is configured to *not* fsync() on all Append() calls
I20260806 19:39:41.438594 27484 log.cc:824] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1: Log is configured to *not* fsync() on all Append() calls
I20260806 19:39:41.442378 27486 tablet_bootstrap.cc:492] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d: No bootstrap required, opened a new log
I20260806 19:39:41.442438 27483 tablet_bootstrap.cc:492] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7: No bootstrap required, opened a new log
I20260806 19:39:41.442730 27484 tablet_bootstrap.cc:492] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1: No bootstrap required, opened a new log
I20260806 19:39:41.442786 27486 ts_tablet_manager.cc:1397] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 19:39:41.443024 27483 ts_tablet_manager.cc:1397] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.004s
I20260806 19:39:41.443024 27484 ts_tablet_manager.cc:1397] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260806 19:39:41.453162 27486 raft_consensus.cc:348] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } } peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } }
I20260806 19:39:41.453740 27486 raft_consensus.cc:374] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:39:41.453858 27486 raft_consensus.cc:729] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc359d524bc4759845f6781e9f6738d, State: Initialized, Role: FOLLOWER
I20260806 19:39:41.454639 27486 consensus_queue.cc:260] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } } peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } }
I20260806 19:39:41.456019 27330 heartbeater.cc:503] Master 127.10.25.62:43233 was elected leader, sending a full tablet report...
I20260806 19:39:41.456182 27484 raft_consensus.cc:348] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } } peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } }
I20260806 19:39:41.456209 27483 raft_consensus.cc:348] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } } peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } }
I20260806 19:39:41.456732 27484 raft_consensus.cc:374] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:39:41.456746 27483 raft_consensus.cc:374] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:39:41.456861 27484 raft_consensus.cc:729] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26a908fd090745318dda758213a0acb1, State: Initialized, Role: FOLLOWER
I20260806 19:39:41.456883 27483 raft_consensus.cc:729] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9da9351acb7e41539f7b2f0ff4da3aa7, State: Initialized, Role: FOLLOWER
I20260806 19:39:41.456954 27486 ts_tablet_manager.cc:1428] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260806 19:39:41.457667 27483 consensus_queue.cc:260] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } } peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } }
I20260806 19:39:41.457679 27484 consensus_queue.cc:260] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } } peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } }
I20260806 19:39:41.463429 27483 ts_tablet_manager.cc:1428] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7: Time spent starting tablet: real 0.020s user 0.015s sys 0.004s
I20260806 19:39:41.463598 27484 ts_tablet_manager.cc:1428] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1: Time spent starting tablet: real 0.020s user 0.022s sys 0.000s
I20260806 19:39:41.464514 27464 heartbeater.cc:503] Master 127.10.25.62:43233 was elected leader, sending a full tablet report...
W20260806 19:39:41.468140 27331 tablet.cc:2369] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:39:41.527582 27466 tablet.cc:2369] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:39:41.637666 27197 tablet.cc:2369] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:39:41.773707 27490 raft_consensus.cc:482] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:39:41.774091 27490 raft_consensus.cc:504] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } } peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } }
I20260806 19:39:41.775936 27490 leader_election.cc:288] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dc359d524bc4759845f6781e9f6738d (127.10.25.2:41159), 9da9351acb7e41539f7b2f0ff4da3aa7 (127.10.25.1:45971)
I20260806 19:39:41.783169 27471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.776420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:41159 (local address 127.10.25.3:51997)
0806 19:39:41.776651 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:41.776675 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:41.776857 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:41.777453 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:41.777457 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:41.777520 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:39:41.777702 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:41.777710 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.778790 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.778796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:41.780409 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:41.780423 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.782381 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.782386 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:41.782562 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.782591 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 19:39:41.782693 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 19:39:41.782817 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"spinlock_wait_cycles":7424}
I20260806 19:39:41.783427 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.776598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:51997 (local address 127.10.25.2:41159)
0806 19:39:41.776826 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:41.776832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:41.776856 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:39:41.777032 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:41.777041 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:41.777161 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:41.777269 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:39:41.777848 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.778630 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.780670 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.782142 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.782351 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.782486 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:41.782763 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 19:39:41.782942 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 19:39:41.784305 27482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.777138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:50107 (local address 127.10.25.1:45971)
0806 19:39:41.777346 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:41.777353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:41.777506 (+ 153us) server_negotiation.cc:407] Connection header received
0806 19:39:41.777708 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:41.777713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:41.777784 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:41.777881 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:39:41.778437 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.779506 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.781437 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.782944 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.783129 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.783257 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:41.783798 (+ 541us) server_negotiation.cc:299] Negotiation successful
0806 19:39:41.783968 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 19:39:41.784308 27492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.776805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:45971 (local address 127.10.25.3:50107)
0806 19:39:41.777380 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:41.777394 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:41.777562 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:41.777898 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:41.777902 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:41.777924 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:39:41.778285 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:41.778294 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.779695 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.779700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:41.781216 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:41.781229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.783227 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.783235 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:41.783557 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.783610 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 19:39:41.783733 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 19:39:41.783901 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":547,"threads_started":1}
I20260806 19:39:41.784902 27489 raft_consensus.cc:482] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:39:41.784782 27284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58e5f292c5b44445bdd7e357bed97121" candidate_uuid: "26a908fd090745318dda758213a0acb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc359d524bc4759845f6781e9f6738d" is_pre_election: true
I20260806 19:39:41.785295 27489 raft_consensus.cc:504] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } } peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } }
I20260806 19:39:41.786163 27151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58e5f292c5b44445bdd7e357bed97121" candidate_uuid: "26a908fd090745318dda758213a0acb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" is_pre_election: true
I20260806 19:39:41.786350 27284 raft_consensus.cc:2454] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26a908fd090745318dda758213a0acb1 in term 0.
I20260806 19:39:41.786721 27151 raft_consensus.cc:2454] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26a908fd090745318dda758213a0acb1 in term 0.
I20260806 19:39:41.787302 27350 leader_election.cc:302] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26a908fd090745318dda758213a0acb1, 5dc359d524bc4759845f6781e9f6738d; no voters:
I20260806 19:39:41.787766 27489 leader_election.cc:288] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26a908fd090745318dda758213a0acb1 (127.10.25.3:36477), 9da9351acb7e41539f7b2f0ff4da3aa7 (127.10.25.1:45971)
I20260806 19:39:41.787792 27490 raft_consensus.cc:2781] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:39:41.787940 27490 raft_consensus.cc:482] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:39:41.788064 27490 raft_consensus.cc:3037] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:39:41.791971 27490 raft_consensus.cc:504] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } } peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } }
I20260806 19:39:41.792797 27490 leader_election.cc:288] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [CANDIDATE]: Term 1 election: Requested vote from peers 5dc359d524bc4759845f6781e9f6738d (127.10.25.2:41159), 9da9351acb7e41539f7b2f0ff4da3aa7 (127.10.25.1:45971)
I20260806 19:39:41.793588 27284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58e5f292c5b44445bdd7e357bed97121" candidate_uuid: "26a908fd090745318dda758213a0acb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc359d524bc4759845f6781e9f6738d"
I20260806 19:39:41.793885 27284 raft_consensus.cc:3037] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:39:41.794476 27151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58e5f292c5b44445bdd7e357bed97121" candidate_uuid: "26a908fd090745318dda758213a0acb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7"
I20260806 19:39:41.794664 27151 raft_consensus.cc:3037] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:39:41.796911 27151 raft_consensus.cc:2454] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26a908fd090745318dda758213a0acb1 in term 1.
I20260806 19:39:41.797590 27350 leader_election.cc:302] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26a908fd090745318dda758213a0acb1, 9da9351acb7e41539f7b2f0ff4da3aa7; no voters:
I20260806 19:39:41.797968 27490 raft_consensus.cc:2781] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:39:41.800787 27284 raft_consensus.cc:2454] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26a908fd090745318dda758213a0acb1 in term 1.
I20260806 19:39:41.801193 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.788369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:44365 (local address 127.10.25.3:36477)
0806 19:39:41.788647 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:41.788657 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:41.789570 (+ 913us) server_negotiation.cc:407] Connection header received
0806 19:39:41.789927 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:41.789931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:41.790038 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:41.790157 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:39:41.790710 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.791729 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.797713 (+ 5984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.799451 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.799597 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.799755 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:41.800615 (+ 860us) server_negotiation.cc:299] Negotiation successful
0806 19:39:41.800836 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 19:39:41.801549 27482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.789017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:58615 (local address 127.10.25.1:45971)
0806 19:39:41.789428 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:41.789434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:41.790049 (+ 615us) server_negotiation.cc:407] Connection header received
0806 19:39:41.790272 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:41.790286 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:41.790361 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:41.790423 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:39:41.791217 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.792335 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.798610 (+ 6275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.800084 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.800194 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.800325 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:41.801053 (+ 728us) server_negotiation.cc:299] Negotiation successful
0806 19:39:41.801218 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288}
I20260806 19:39:41.802232 27494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.787958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:36477 (local address 127.10.25.2:44365)
0806 19:39:41.789395 (+ 1437us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:41.789424 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:41.789738 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:41.790177 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:41.790182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:41.790215 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:39:41.790549 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:41.790559 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.791947 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.791951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:41.797445 (+ 5494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:41.797469 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.800047 (+ 2578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.800057 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:41.800337 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.800393 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 19:39:41.801365 (+ 972us) client_negotiation.cc:240] Negotiation successful
0806 19:39:41.801612 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1246,"mutex_wait_us":38,"thread_start_us":1655,"threads_started":1}
I20260806 19:39:41.802990 27493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.788272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:45971 (local address 127.10.25.2:58615)
0806 19:39:41.789923 (+ 1651us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:41.789942 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:39:41.790090 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:39:41.790637 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:39:41.790641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:39:41.790671 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:39:41.791026 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:41.791036 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.792521 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.792526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:41.798047 (+ 5521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:39:41.798070 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.800578 (+ 2508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.800586 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:39:41.800801 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.800846 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 19:39:41.801808 (+ 962us) client_negotiation.cc:240] Negotiation successful
0806 19:39:41.801977 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1530,"mutex_wait_us":30,"thread_start_us":781,"threads_started":1}
I20260806 19:39:41.803452 27418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58e5f292c5b44445bdd7e357bed97121" candidate_uuid: "5dc359d524bc4759845f6781e9f6738d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26a908fd090745318dda758213a0acb1" is_pre_election: true
I20260806 19:39:41.803655 27151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58e5f292c5b44445bdd7e357bed97121" candidate_uuid: "5dc359d524bc4759845f6781e9f6738d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" is_pre_election: true
I20260806 19:39:41.804060 27151 raft_consensus.cc:2379] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5dc359d524bc4759845f6781e9f6738d in current term 1: Already voted for candidate 26a908fd090745318dda758213a0acb1 in this term.
I20260806 19:39:41.806264 27490 raft_consensus.cc:686] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [term 1 LEADER]: Becoming Leader. State: Replica: 26a908fd090745318dda758213a0acb1, State: Running, Role: LEADER
I20260806 19:39:41.806973 27490 consensus_queue.cc:237] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } } peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } }
I20260806 19:39:41.812433 27219 leader_election.cc:302] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5dc359d524bc4759845f6781e9f6738d; no voters: 26a908fd090745318dda758213a0acb1, 9da9351acb7e41539f7b2f0ff4da3aa7
I20260806 19:39:41.813088 27489 raft_consensus.cc:2726] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 19:39:41.813853 27008 catalog_manager.cc:5495] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26a908fd090745318dda758213a0acb1 (127.10.25.3). New cstate: current_term: 1 leader_uuid: "26a908fd090745318dda758213a0acb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26a908fd090745318dda758213a0acb1" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 36477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 } health_report { overall_health: UNKNOWN } } }
I20260806 19:39:41.870075 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.862567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36656 (local address 127.10.25.3:36477)
0806 19:39:41.862814 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:41.862823 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:41.863191 (+ 368us) server_negotiation.cc:407] Connection header received
0806 19:39:41.863392 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:41.863397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:41.863578 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:41.863733 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:39:41.864083 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.864927 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.867227 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.867674 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.867796 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.868505 (+ 709us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:39:41.869069 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:39:41.869228 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:41.869555 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 19:39:41.869662 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 19:39:41.880203 27151 raft_consensus.cc:1261] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 1 FOLLOWER]: Refusing update from remote peer 26a908fd090745318dda758213a0acb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:39:41.880262 27284 raft_consensus.cc:1261] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d [term 1 FOLLOWER]: Refusing update from remote peer 26a908fd090745318dda758213a0acb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:39:41.881404 27495 consensus_queue.cc:1035] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dc359d524bc4759845f6781e9f6738d" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:39:41.881950 27490 consensus_queue.cc:1035] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9da9351acb7e41539f7b2f0ff4da3aa7" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 45971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:39:41.897821 27500 mvcc.cc:205] Tried to move back new op lower bound from 7315641064968790016 to 7315641064685973504. Current Snapshot: MvccSnapshot[applied={T|T < 7315641064968790016}]
I20260806 19:39:41.925396 27008 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:39:41.925787 27008 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:39:41.930419 27008 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Sending DeleteTablet for 3 replicas of tablet 58e5f292c5b44445bdd7e357bed97121
I20260806 19:39:41.932520 27264 tablet_service.cc:1514] Processing DeleteTablet for tablet 58e5f292c5b44445bdd7e357bed97121 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:39:41 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:35884
I20260806 19:39:41.933344 27398 tablet_service.cc:1514] Processing DeleteTablet for tablet 58e5f292c5b44445bdd7e357bed97121 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:39:41 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:36640
I20260806 19:39:41.933637 27509 tablet_replica.cc:330] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d: stopping tablet replica
I20260806 19:39:41.934038 27509 raft_consensus.cc:2229] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:39:41.934453 27130 tablet_service.cc:1514] Processing DeleteTablet for tablet 58e5f292c5b44445bdd7e357bed97121 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:39:41 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:55392
I20260806 19:39:41.934610 27509 raft_consensus.cc:2258] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:39:41.935154 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 27046
I20260806 19:39:41.939248 27509 ts_tablet_manager.cc:1905] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:39:41.943953 27510 tablet_replica.cc:330] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1: stopping tablet replica
I20260806 19:39:41.944777 27510 raft_consensus.cc:2229] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:39:41.945830 27510 raft_consensus.cc:2258] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:39:41.949435 27510 ts_tablet_manager.cc:1905] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:39:41.950389 27509 ts_tablet_manager.cc:1918] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:39:41.950639 27509 log.cc:1196] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/58e5f292c5b44445bdd7e357bed97121
I20260806 19:39:41.951395 27509 ts_tablet_manager.cc:1939] T 58e5f292c5b44445bdd7e357bed97121 P 5dc359d524bc4759845f6781e9f6738d: Deleting consensus metadata
I20260806 19:39:41.954123 26993 catalog_manager.cc:4841] TS 5dc359d524bc4759845f6781e9f6738d (127.10.25.2:41159): tablet 58e5f292c5b44445bdd7e357bed97121 (table test-table [id=267f00aa7a5446d69c053c9914637764]) successfully deleted
I20260806 19:39:41.955098 27511 tablet_replica.cc:330] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7: stopping tablet replica
I20260806 19:39:41.955766 27511 raft_consensus.cc:2229] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:39:41.956624 27511 raft_consensus.cc:2258] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:39:41.962554 27511 ts_tablet_manager.cc:1905] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:39:41.966533 27510 ts_tablet_manager.cc:1918] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:39:41.971927 27510 log.cc:1196] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/58e5f292c5b44445bdd7e357bed97121
I20260806 19:39:41.973071 27510 ts_tablet_manager.cc:1939] T 58e5f292c5b44445bdd7e357bed97121 P 26a908fd090745318dda758213a0acb1: Deleting consensus metadata
I20260806 19:39:41.976006 26995 catalog_manager.cc:4841] TS 26a908fd090745318dda758213a0acb1 (127.10.25.3:36477): tablet 58e5f292c5b44445bdd7e357bed97121 (table test-table [id=267f00aa7a5446d69c053c9914637764]) successfully deleted
I20260806 19:39:41.978056 27482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:41.936438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46682 (local address 127.10.25.1:45971)
0806 19:39:41.936762 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:41.936769 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:41.955899 (+ 19130us) server_negotiation.cc:407] Connection header received
0806 19:39:41.956121 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:41.956128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:41.956234 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:41.956374 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:41.959318 (+ 2944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.960474 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.963999 (+ 3525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:41.964834 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:41.964974 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:41.969620 (+ 4646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:41.969644 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:41.969672 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:41.969746 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:41.974042 (+ 4296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:41.974980 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:41.974988 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:41.974991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:41.975144 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:41.975785 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:41.975790 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:41.975793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:41.976424 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:41.976757 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:41.977342 (+ 585us) server_negotiation.cc:299] Negotiation successful
0806 19:39:41.977683 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260806 19:39:41.978770 27191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:39:42.112397 27511 ts_tablet_manager.cc:1918] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:39:42.112741 27511 log.cc:1196] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/58e5f292c5b44445bdd7e357bed97121
I20260806 19:39:42.113121 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 27046
W20260806 19:39:42.133621 26993 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.25.1:45971 (error 108)
I20260806 19:39:42.134452 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 27200
I20260806 19:39:42.136276 27515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:42.135302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:45971 (local address 127.0.0.1:46690)
0806 19:39:42.135886 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:42.136030 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:45971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":312,"threads_started":1}
W20260806 19:39:42.136623 26993 catalog_manager.cc:4584] TS 9da9351acb7e41539f7b2f0ff4da3aa7 (127.10.25.1:45971): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 58e5f292c5b44445bdd7e357bed97121: Network error: Client connection negotiation failed: client connection to 127.10.25.1:45971: connect: Connection refused (error 111)
I20260806 19:39:42.152597 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:42.135300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60378 (local address 127.10.25.2:41159)
0806 19:39:42.135488 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:42.135494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:42.135781 (+ 287us) server_negotiation.cc:407] Connection header received
0806 19:39:42.135859 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:42.135866 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:42.135956 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:42.136091 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:42.138091 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:42.138884 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:42.140777 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:42.141209 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:42.141319 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:42.145038 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:42.145057 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:42.145074 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:42.145118 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:42.148457 (+ 3339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:42.149607 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:42.149619 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:42.149626 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:42.149849 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:42.150354 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:42.150358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:42.150361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:42.150803 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:42.151054 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:42.151912 (+ 858us) server_negotiation.cc:299] Negotiation successful
0806 19:39:42.152273 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 19:39:42.153687 27325 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:39:42.190325 27515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:42.189847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:45971 (local address 127.0.0.1:46694)
0806 19:39:42.190072 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:42.190171 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:45971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 19:39:42.191610 27515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:42.191201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:45971 (local address 127.0.0.1:46710)
0806 19:39:42.191358 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:42.191468 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:45971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 19:39:42.252472 27515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:42.251898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:45971 (local address 127.0.0.1:46714)
0806 19:39:42.252117 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:42.252246 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:45971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 19:39:42.253767 27515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:42.253287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:45971 (local address 127.0.0.1:46722)
0806 19:39:42.253454 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:42.253604 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:45971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 19:39:42.301548 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 27200
I20260806 19:39:42.325994 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 27334
I20260806 19:39:42.335144 27515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:42.334576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:45971 (local address 127.0.0.1:46736)
0806 19:39:42.334846 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:42.334955 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:45971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 19:39:42.336627 27515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:42.336036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:45971 (local address 127.0.0.1:46752)
0806 19:39:42.336300 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:42.336410 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:45971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260806 19:39:42.345896 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:42.326943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60608 (local address 127.10.25.3:36477)
0806 19:39:42.327202 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 19:39:42.327208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:39:42.327232 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:39:42.327538 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:39:42.327544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:39:42.327663 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:39:42.327789 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:39:42.329527 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:42.330641 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:42.332258 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:39:42.333070 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:39:42.333220 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:39:42.337226 (+ 4006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:39:42.337243 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:39:42.337264 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:39:42.337342 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:39:42.341922 (+ 4580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:42.343166 (+ 1244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:42.343176 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:42.343180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:42.343367 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:39:42.343874 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:39:42.343879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:39:42.343882 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:39:42.344429 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:39:42.344791 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0806 19:39:42.345198 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 19:39:42.345536 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 19:39:42.347352 27459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:39:42.479789 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 27334
I20260806 19:39:42.504122 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 26976
I20260806 19:39:42.505432 27039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:39:42.622939 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 26976
I20260806 19:39:42.623073 27515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:39:42.622369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:45971 (local address 127.0.0.1:46764)
0806 19:39:42.622714 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 19:39:42.622838 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:45971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260806 19:40:02.648555 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:43233
--webserver_interface=127.10.25.62
--webserver_port=36495
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:43233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:03.030591 27519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:03.030998 27519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:03.031188 27519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:03.038889 27519 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 19:40:03.039086 27519 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 19:40:03.039191 27519 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 19:40:03.039287 27519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:40:03.039394 27519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:03.039515 27519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:03.039609 27519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:40:03.039690 27519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:40:03.050256 27519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:43233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:43233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=36495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:03.051800 27519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:03.053355 27519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:03.067677 27527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:03.067757 27524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:03.067713 27525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:03.068544 27519 server_base.cc:1034] running on GCE node
I20260806 19:40:03.069404 27519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:03.071076 27519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:03.072394 27519 hybrid_clock.cc:648] HybridClock initialized: now 1786045203072327 us; error 44 us; skew 500 ppm
I20260806 19:40:03.072842 27519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:03 dist-test-slave-ss4n krb5kdc[26417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045203, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:03.080853 27519 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:40:03.083056 27519 webserver.cc:460] Webserver started at http://127.10.25.62:36495/ using document root <none> and password file <none>
I20260806 19:40:03.083927 27519 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:03.084092 27519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:03.090981 27519 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 19:40:03.095548 27534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:03.097909 27519 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260806 19:40:03.098208 27519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "9d61ccd09113402aa039f1960b2dc3a6"
format_stamp: "Formatted at 2026-08-06 19:39:15 on dist-test-slave-ss4n"
server_key: "db23c1de69d2ef087708a7a5d487d557"
server_key_iv: "042c540f263c0edf0c53049dc0757b21"
server_key_version: "encryptionkey@0"
I20260806 19:40:03.099512 27519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:03.120388 27519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:03.125434 27519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:03.125859 27519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:03.151206 27519 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:43233
I20260806 19:40:03.151218 27587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:43233 every 8 connection(s)
I20260806 19:40:03.153955 27519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:40:03.163591 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 27519
I20260806 19:40:03.163854 27588 sys_catalog.cc:263] Verifying existing consensus state
I20260806 19:40:03.164652 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:45971
--local_ip_for_outbound_sockets=127.10.25.1
--tserver_master_addrs=127.10.25.62:43233
--webserver_port=36833
--webserver_interface=127.10.25.1
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:40:03.168154 27588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Bootstrap starting.
I20260806 19:40:03.193725 27588 log.cc:824] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:03.213018 27588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 19:40:03.213748 27588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Bootstrap complete.
I20260806 19:40:03.223281 27588 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } }
I20260806 19:40:03.224313 27588 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d61ccd09113402aa039f1960b2dc3a6, State: Initialized, Role: FOLLOWER
I20260806 19:40:03.225061 27588 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } }
I20260806 19:40:03.225333 27588 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:40:03.225472 27588 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:40:03.225749 27588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 2 FOLLOWER]: Advancing to term 3
I20260806 19:40:03.230201 27588 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } }
I20260806 19:40:03.230818 27588 leader_election.cc:302] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d61ccd09113402aa039f1960b2dc3a6; no voters:
I20260806 19:40:03.231611 27588 leader_election.cc:288] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 19:40:03.231779 27592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 19:40:03.233047 27592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [term 3 LEADER]: Becoming Leader. State: Replica: 9d61ccd09113402aa039f1960b2dc3a6, State: Running, Role: LEADER
I20260806 19:40:03.233731 27592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } }
I20260806 19:40:03.234215 27588 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:40:03.236635 27593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9d61ccd09113402aa039f1960b2dc3a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } } }
I20260806 19:40:03.237138 27593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:03.237785 27594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d61ccd09113402aa039f1960b2dc3a6. Latest consensus state: current_term: 3 leader_uuid: "9d61ccd09113402aa039f1960b2dc3a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d61ccd09113402aa039f1960b2dc3a6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43233 } } }
I20260806 19:40:03.238157 27594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6 [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:03.243767 27598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:40:03.254670 27598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:40:03.256803 27598 catalog_manager.cc:1259] Loaded cluster ID: eb8501212f7b48e4a6bfe0d15f50defe
I20260806 19:40:03.256965 27598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:40:03.261405 27609 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 19:40:03.262466 27598 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:40:03.263985 27598 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Loaded TSK: 0
I20260806 19:40:03.264161 27598 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9d61ccd09113402aa039f1960b2dc3a6: Loaded TSK: 1
F20260806 19:40:03.265022 27598 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f06019049ad google::LogMessage::Fail() at ??:0
@ 0x7f060190788a google::LogMessage::SendToLog() at ??:0
@ 0x7f06019044de google::LogMessage::Flush() at ??:0
@ 0x7f0601905a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f06056687d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f060565773e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f06173331c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f0617332783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f06173be2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f06173be0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f06174085e9 std::function<>::operator()() at ??:0
@ 0x7f0617336cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f0617334c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f06173bc610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f06173bc439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0604550345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f060455391c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f0604553799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f060452eeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f06078086db start_thread at ??:0
@ 0x7f060079e61f clone at ??:0
W20260806 19:40:03.625640 27590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:03.625975 27590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:03.626080 27590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:03.633911 27590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:03.634052 27590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:03.634174 27590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:40:03.644774 27590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:45971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=36833
--tserver_master_addrs=127.10.25.62:43233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:03.646241 27590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:03.647578 27590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:03.662228 27635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:03.662429 27636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:03.662748 27638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:03.663470 27590 server_base.cc:1034] running on GCE node
I20260806 19:40:03.664251 27590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:03.665410 27590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:03.666765 27590 hybrid_clock.cc:648] HybridClock initialized: now 1786045203666714 us; error 41 us; skew 500 ppm
I20260806 19:40:03.667158 27590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:03 dist-test-slave-ss4n krb5kdc[26417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045203, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:03.671111 27590 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:40:03.672847 27590 webserver.cc:460] Webserver started at http://127.10.25.1:36833/ using document root <none> and password file <none>
I20260806 19:40:03.673753 27590 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:03.673908 27590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:03.680361 27590 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260806 19:40:03.683866 27645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:03.685803 27590 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:40:03.686071 27590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "9da9351acb7e41539f7b2f0ff4da3aa7"
format_stamp: "Formatted at 2026-08-06 19:39:16 on dist-test-slave-ss4n"
server_key: "8b85c43e172ff9ac7c1a0c8c07193e2c"
server_key_iv: "fee25853912f5dbc3ed52b790e402fd8"
server_key_version: "encryptionkey@0"
I20260806 19:40:03.687318 27590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:03.715557 27590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:03.721716 27590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:03.722071 27590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:03.723959 27590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:03.726420 27652 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260806 19:40:03.732491 27652 ts_tablet_manager.cc:1852] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260806 19:40:03.733741 27652 ts_tablet_manager.cc:1905] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:03.742766 27652 ts_tablet_manager.cc:1918] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:40:03.743477 27652 ts_tablet_manager.cc:1939] T 58e5f292c5b44445bdd7e357bed97121 P 9da9351acb7e41539f7b2f0ff4da3aa7: Deleting consensus metadata
I20260806 19:40:03.744120 27590 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260806 19:40:03.744287 27590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.018s user 0.001s sys 0.000s
I20260806 19:40:03.744441 27590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:03.744546 27590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:03.785643 27590 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:45971
I20260806 19:40:03.785882 27760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:45971 every 8 connection(s)
I20260806 19:40:03.787348 27590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:40:03.793140 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 27590
I20260806 19:40:03.793824 27763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:03.791004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:55451)
0806 19:40:03.791939 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:03.792251 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":385,"threads_started":1}
I20260806 19:40:03.794073 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:41159
--local_ip_for_outbound_sockets=127.10.25.2
--tserver_master_addrs=127.10.25.62:43233
--webserver_port=46557
--webserver_interface=127.10.25.2
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:03.794806 27761 heartbeater.cc:644] Failed to heartbeat to 127.10.25.62:43233 (0 consecutive failures): Network error: Failed to ping master at 127.10.25.62:43233: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
I20260806 19:40:03.795997 27763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:03.795453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:34371)
0806 19:40:03.795667 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:03.795812 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 19:40:03.797219 27763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:03.796702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:56365)
0806 19:40:03.796900 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:03.797016 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 19:40:03.797510 27761 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 19:40:04.187084 27764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:04.187417 27764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:04.187562 27764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:04.195634 27764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:04.195779 27764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:04.195963 27764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:40:04.212639 27764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:41159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=46557
--tserver_master_addrs=127.10.25.62:43233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:04.214533 27764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:04.216930 27764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:04.238423 27770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:04.238723 27769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:04.238751 27772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:04.241679 27764 server_base.cc:1034] running on GCE node
I20260806 19:40:04.243175 27764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:04.245297 27764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:04.246645 27764 hybrid_clock.cc:648] HybridClock initialized: now 1786045204246572 us; error 116 us; skew 500 ppm
I20260806 19:40:04.247210 27764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:04 dist-test-slave-ss4n krb5kdc[26417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045204, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:04.252933 27764 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:40:04.255553 27764 webserver.cc:460] Webserver started at http://127.10.25.2:46557/ using document root <none> and password file <none>
I20260806 19:40:04.256750 27764 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:04.256942 27764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:04.265091 27764 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.001s
I20260806 19:40:04.268530 27780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:04.270548 27764 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:40:04.270780 27764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "5dc359d524bc4759845f6781e9f6738d"
format_stamp: "Formatted at 2026-08-06 19:39:17 on dist-test-slave-ss4n"
server_key: "45cffeac6f03a84ff047ad823c0b0e6d"
server_key_iv: "b855018a4064c41afe4f9f2ad64d95e8"
server_key_version: "encryptionkey@0"
I20260806 19:40:04.271936 27764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:04.309868 27764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:04.315214 27764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:04.315503 27764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:04.316687 27764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:04.318408 27764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:04.318517 27764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:04.318602 27764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:04.318660 27764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:04.360965 27764 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:41159
I20260806 19:40:04.361030 27894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:41159 every 8 connection(s)
I20260806 19:40:04.362789 27764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:40:04.368778 27897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:04.366611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:57989)
0806 19:40:04.367561 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:04.367814 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":406,"threads_started":1}
W20260806 19:40:04.369737 27895 heartbeater.cc:644] Failed to heartbeat to 127.10.25.62:43233 (0 consecutive failures): Network error: Failed to ping master at 127.10.25.62:43233: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
I20260806 19:40:04.370827 27897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:04.370382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:44499)
0806 19:40:04.370592 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:04.370682 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 19:40:04.372066 27897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:04.371605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:48487)
0806 19:40:04.371786 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:04.371921 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260806 19:40:04.372398 27895 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 19:40:04.372954 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 27764
I20260806 19:40:04.373708 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:36477
--local_ip_for_outbound_sockets=127.10.25.3
--tserver_master_addrs=127.10.25.62:43233
--webserver_port=40743
--webserver_interface=127.10.25.3
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:04.762701 27898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:04.763110 27898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:04.763291 27898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:04.771752 27898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:04.771986 27898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:04.772197 27898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:40:04.783927 27898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:36477
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=40743
--tserver_master_addrs=127.10.25.62:43233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:04.785455 27898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:04.787113 27898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 19:40:04.799480 27907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:04.798484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:44763)
0806 19:40:04.799089 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:04.799188 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":302,"threads_started":1}
W20260806 19:40:04.804234 27903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:04.804284 27904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:04.804260 27906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:04.805740 27898 server_base.cc:1034] running on GCE node
I20260806 19:40:04.806571 27898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:04.807754 27898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:04.809069 27898 hybrid_clock.cc:648] HybridClock initialized: now 1786045204808981 us; error 77 us; skew 500 ppm
I20260806 19:40:04.809581 27898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:04 dist-test-slave-ss4n krb5kdc[26417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045204, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:04.813241 27898 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:40:04.815047 27898 webserver.cc:460] Webserver started at http://127.10.25.3:40743/ using document root <none> and password file <none>
I20260806 19:40:04.815824 27898 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:04.815953 27898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:04.824625 27898 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.005s
I20260806 19:40:04.828867 27914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:04.830771 27898 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260806 19:40:04.830969 27898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "26a908fd090745318dda758213a0acb1"
format_stamp: "Formatted at 2026-08-06 19:39:17 on dist-test-slave-ss4n"
server_key: "dfb314d0f2acc657b313e0e6ce440d49"
server_key_iv: "f485b6ad44a4bd58ce2643ab73c0dba9"
server_key_version: "encryptionkey@0"
I20260806 19:40:04.832082 27898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:04.865736 27898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:04.871520 27898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:04.871833 27898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:04.873096 27898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:04.874886 27898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:04.875005 27898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:04.875099 27898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:04.875171 27898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:04.917677 27898 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:36477
I20260806 19:40:04.917747 28028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:36477 every 8 connection(s)
I20260806 19:40:04.919371 27898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:40:04.921293 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 27898
I20260806 19:40:04.925242 28032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:04.923195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:48909)
0806 19:40:04.924066 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:04.924297 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":361,"threads_started":1}
W20260806 19:40:04.926100 28029 heartbeater.cc:644] Failed to heartbeat to 127.10.25.62:43233 (0 consecutive failures): Network error: Failed to ping master at 127.10.25.62:43233: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
I20260806 19:40:04.927270 28032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:04.926814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:40573)
0806 19:40:04.927055 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:04.927136 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 19:40:04.928435 28032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:04.927986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:49801)
0806 19:40:04.928189 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:04.928296 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260806 19:40:04.928854 28029 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 19:40:05.374359 28033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:05.373229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:33877)
0806 19:40:05.373955 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:05.374096 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":388,"threads_started":1}
I20260806 19:40:05.801433 28034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:05.800623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:38217)
0806 19:40:05.801141 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:05.801238 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":266,"threads_started":1}
I20260806 19:40:05.931000 28035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:05.930000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:49427)
0806 19:40:05.930612 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:05.930726 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":305,"threads_started":1}
I20260806 19:40:06.376667 28036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:06.375595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:50845)
0806 19:40:06.376260 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:06.376368 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":353,"threads_started":1}
I20260806 19:40:06.803555 28037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:06.802650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:41047)
0806 19:40:06.803238 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:06.803350 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":295,"threads_started":1}
I20260806 19:40:06.933148 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:06.932292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:58961)
0806 19:40:06.932862 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:06.932958 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":311,"threads_started":1}
I20260806 19:40:07.378608 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:07.377781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:41845)
0806 19:40:07.378338 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:07.378419 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":326,"threads_started":1}
I20260806 19:40:07.806170 28040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:07.805231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:59391)
0806 19:40:07.805880 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:07.805972 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":403,"threads_started":1}
I20260806 19:40:07.935918 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:07.934774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:42509)
0806 19:40:07.935503 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:07.935675 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":452,"threads_started":1}
I20260806 19:40:08.380610 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:08.379734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:37073)
0806 19:40:08.380313 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:08.380412 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":321,"threads_started":1}
I20260806 19:40:08.808619 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:08.807350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:33521)
0806 19:40:08.808115 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:08.808246 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":396,"threads_started":1}
I20260806 19:40:08.938119 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:08.937222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:49955)
0806 19:40:08.937831 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:08.937917 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":382,"threads_started":1}
I20260806 19:40:09.382838 28045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:09.381977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:48439)
0806 19:40:09.382520 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:09.382643 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":276,"threads_started":1}
I20260806 19:40:09.810976 28046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:09.809840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:34857)
0806 19:40:09.810569 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:09.810700 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":362,"threads_started":1}
I20260806 19:40:09.940407 28047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:09.939220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:45779)
0806 19:40:09.940013 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:09.940151 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":409,"threads_started":1}
I20260806 19:40:10.385522 28048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:10.384467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:48325)
0806 19:40:10.385190 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:10.385310 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":374,"threads_started":1}
I20260806 19:40:10.813807 28049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:10.812425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:55283)
0806 19:40:10.813253 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:10.813381 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":358,"threads_started":1}
I20260806 19:40:10.942611 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:10.941735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:54649)
0806 19:40:10.942277 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:10.942382 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":282,"threads_started":1}
I20260806 19:40:11.388217 28051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:11.386916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:49895)
0806 19:40:11.387854 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:11.387971 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":760,"thread_start_us":471,"threads_started":1}
I20260806 19:40:11.816583 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:11.815114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:48663)
0806 19:40:11.815898 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:11.816042 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":409,"threads_started":1}
I20260806 19:40:11.944773 28053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:11.943766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:36485)
0806 19:40:11.944433 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:11.944536 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":362,"threads_started":1}
I20260806 19:40:12.390792 28054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:12.389706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:44555)
0806 19:40:12.390511 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:12.390606 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":371,"threads_started":1}
I20260806 19:40:12.818639 28055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:12.817749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:49387)
0806 19:40:12.818322 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:12.818427 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":299,"threads_started":1}
I20260806 19:40:12.946681 28056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:12.945882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:49937)
0806 19:40:12.946390 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:12.946480 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":269,"threads_started":1}
I20260806 19:40:13.393265 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:13.392167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:58899)
0806 19:40:13.392845 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:13.393002 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":363,"threads_started":1}
I20260806 19:40:13.820844 28058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:13.819892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:40469)
0806 19:40:13.820451 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:13.820615 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":311,"threads_started":1}
I20260806 19:40:13.948881 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:13.947954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:55169)
0806 19:40:13.948539 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:13.948692 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":288,"threads_started":1}
I20260806 19:40:14.395948 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:14.395001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:50901)
0806 19:40:14.395641 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:14.395743 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":354,"threads_started":1}
I20260806 19:40:14.822827 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:14.821989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:52941)
0806 19:40:14.822546 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:14.822641 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":296,"threads_started":1}
I20260806 19:40:14.951401 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:14.950448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:45207)
0806 19:40:14.951089 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:14.951193 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":331,"threads_started":1}
I20260806 19:40:15.398456 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:15.397325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:54465)
0806 19:40:15.398106 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:15.398242 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":493,"threads_started":1}
I20260806 19:40:15.825314 28065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:15.824187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:32963)
0806 19:40:15.824942 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:15.825077 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":400,"threads_started":1}
I20260806 19:40:15.954178 28066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:15.952911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:34253)
0806 19:40:15.953824 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:15.953958 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":589,"threads_started":1}
I20260806 19:40:16.401001 28067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:16.400063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:60563)
0806 19:40:16.400709 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:16.400821 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":392,"threads_started":1}
I20260806 19:40:16.829309 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:16.827545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:53765)
0806 19:40:16.828873 (+ 1328us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:16.829060 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1044,"thread_start_us":817,"threads_started":1}
I20260806 19:40:16.956449 28069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:16.955522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:39657)
0806 19:40:16.956125 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:16.956232 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":325,"threads_started":1}
I20260806 19:40:17.403851 28070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:17.402564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:35451)
0806 19:40:17.403373 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:17.403569 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":382,"threads_started":1}
I20260806 19:40:17.832080 28071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:17.830620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:55795)
0806 19:40:17.831596 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:17.831751 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":793,"thread_start_us":307,"threads_started":1}
I20260806 19:40:17.958860 28072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:17.957923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:48547)
0806 19:40:17.958478 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:17.958663 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":310,"threads_started":1}
I20260806 19:40:18.406716 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:18.405381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:55567)
0806 19:40:18.406233 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:18.406442 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":447,"threads_started":1}
I20260806 19:40:18.834990 28074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:18.833876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:57051)
0806 19:40:18.834555 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:18.834787 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":347,"threads_started":1}
I20260806 19:40:18.961179 28075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:18.960268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:47957)
0806 19:40:18.960856 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:18.960955 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":294,"threads_started":1}
I20260806 19:40:19.409718 28076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:19.408425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.2:38637)
0806 19:40:19.409180 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:19.409338 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":397,"threads_started":1}
I20260806 19:40:19.837713 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:19.836292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.1:48503)
0806 19:40:19.837339 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:19.837438 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":781,"threads_started":1}
I20260806 19:40:19.933929 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 27590
I20260806 19:40:19.963837 28079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:19.962931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43233 (local address 127.10.25.3:50253)
0806 19:40:19.963537 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:19.963640 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.62:43233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":331,"threads_started":1}
I20260806 19:40:19.966861 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:19.935786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47958 (local address 127.10.25.1:45971)
0806 19:40:19.936684 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:19.936708 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:19.936767 (+ 59us) server_negotiation.cc:407] Connection header received
0806 19:40:19.938917 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:19.938931 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:19.939449 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:19.939787 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:19.941891 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:19.944181 (+ 2290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:19.946307 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:19.947390 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:19.947641 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:19.956859 (+ 9218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:19.956871 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:19.956891 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:19.956953 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:19.961134 (+ 4181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:19.962497 (+ 1363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:19.962510 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:19.962522 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:19.962857 (+ 335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:19.963631 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:19.963637 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:19.963640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:19.964480 (+ 840us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:19.964857 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:19.965641 (+ 784us) server_negotiation.cc:299] Negotiation successful
0806 19:40:19.966321 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":465,"threads_started":1}
I20260806 19:40:19.969911 27756 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:20.092098 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 27590
I20260806 19:40:20.115893 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 27764
I20260806 19:40:20.142455 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:20.117232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42232 (local address 127.10.25.2:41159)
0806 19:40:20.117873 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:20.117887 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:20.117925 (+ 38us) server_negotiation.cc:407] Connection header received
0806 19:40:20.119465 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:20.119475 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:20.119927 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:20.120246 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:20.122043 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:20.123662 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:20.125321 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:20.126602 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:20.126877 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:20.131505 (+ 4628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:20.131519 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:20.131546 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:20.131623 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:20.136820 (+ 5197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:20.138332 (+ 1512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:20.138344 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:20.138353 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:20.138644 (+ 291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:20.139222 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:20.139227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:20.139229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:20.140212 (+ 983us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:20.140582 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:20.141448 (+ 866us) server_negotiation.cc:299] Negotiation successful
0806 19:40:20.142099 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":369,"threads_started":1}
I20260806 19:40:20.144479 27890 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:20.245792 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 27764
I20260806 19:40:20.268905 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 27898
I20260806 19:40:20.293037 28083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:20.270095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37770 (local address 127.10.25.3:36477)
0806 19:40:20.270787 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:20.270802 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:20.270849 (+ 47us) server_negotiation.cc:407] Connection header received
0806 19:40:20.272754 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:20.272766 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:20.273258 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:20.273829 (+ 571us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:20.275708 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:20.277202 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:20.278899 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:20.279945 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:20.280213 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:20.283785 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:20.283796 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:20.283816 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:20.283876 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:20.288808 (+ 4932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:20.289929 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:20.289938 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:20.289946 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:20.290165 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:20.290614 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:20.290618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:20.290620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:20.291260 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:20.291661 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:20.292158 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 19:40:20.292709 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":333,"threads_started":1}
I20260806 19:40:20.294952 28024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:20.403092 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 27898
2026-08-06T19:40:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (66701 ms)
[----------] 30 tests from SecurityITest (358562 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:40:20 dist-test-slave-ss4n krb5kdc[28087](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:40:20 dist-test-slave-ss4n krb5kdc[28087](info): set up 2 sockets
Aug 06 19:40:20 dist-test-slave-ss4n krb5kdc[28087](info): commencing operation
krb5kdc: starting...
W20260806 19:40:21.738586 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.047s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:40:21 dist-test-slave-ss4n krb5kdc[28087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:40:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:40:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:40:22.124617 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:39071
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:38069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:39071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:22.539074 28100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:22.539426 28100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:22.539523 28100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:22.547415 28100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:40:22.547564 28100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:22.547622 28100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:22.547683 28100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:40:22.547722 28100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:40:22.558270 28100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:38069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:39071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:39071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:22.559679 28100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:22.561121 28100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:22.575278 28106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:22.575358 28108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:22.575335 28105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:22.576006 28100 server_base.cc:1034] running on GCE node
I20260806 19:40:22.577068 28100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:22.578903 28100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:22.580204 28100 hybrid_clock.cc:648] HybridClock initialized: now 1786045222580151 us; error 47 us; skew 500 ppm
I20260806 19:40:22.580684 28100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:22 dist-test-slave-ss4n krb5kdc[28087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045222, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:22.585202 28100 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:40:22.587280 28100 webserver.cc:460] Webserver started at http://127.10.25.62:34915/ using document root <none> and password file <none>
I20260806 19:40:22.588110 28100 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:22.588285 28100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:22.588637 28100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:22.592141 28100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "8377841e1e6541399143bd371d22d852"
format_stamp: "Formatted at 2026-08-06 19:40:22 on dist-test-slave-ss4n"
server_key: "d4663c5efcbe313f9cf14a9f1a279df1"
server_key_iv: "62dea55ba4354f706938b6d8f0f18c5d"
server_key_version: "encryptionkey@0"
I20260806 19:40:22.593050 28100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "8377841e1e6541399143bd371d22d852"
format_stamp: "Formatted at 2026-08-06 19:40:22 on dist-test-slave-ss4n"
server_key: "d4663c5efcbe313f9cf14a9f1a279df1"
server_key_iv: "62dea55ba4354f706938b6d8f0f18c5d"
server_key_version: "encryptionkey@0"
I20260806 19:40:22.599983 28100 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.009s
I20260806 19:40:22.605355 28115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:22.607719 28100 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.003s
I20260806 19:40:22.608088 28100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "8377841e1e6541399143bd371d22d852"
format_stamp: "Formatted at 2026-08-06 19:40:22 on dist-test-slave-ss4n"
server_key: "d4663c5efcbe313f9cf14a9f1a279df1"
server_key_iv: "62dea55ba4354f706938b6d8f0f18c5d"
server_key_version: "encryptionkey@0"
I20260806 19:40:22.608366 28100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:22.624500 28100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:22.630436 28100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:22.630800 28100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:22.657167 28100 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:39071
I20260806 19:40:22.657213 28168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:39071 every 8 connection(s)
I20260806 19:40:22.659202 28100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:40:22.663895 28169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:22.664110 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 28100
I20260806 19:40:22.664319 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:40:22.665211 10340 external_mini_cluster.cc:1428] Setting key fe4c1674d6941b15b6db60b5300db7db
I20260806 19:40:22.681063 28169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852: Bootstrap starting.
Aug 06 19:40:22 dist-test-slave-ss4n krb5kdc[28087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:40:22.686196 28169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:22.688144 28169 log.cc:824] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:22.692376 28169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852: No bootstrap required, opened a new log
I20260806 19:40:22.696660 28172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:22.668161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41236 (local address 127.10.25.62:39071)
0806 19:40:22.669192 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:22.669207 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:22.669261 (+ 54us) server_negotiation.cc:407] Connection header received
0806 19:40:22.671902 (+ 2641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:22.671933 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:22.672465 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:22.673066 (+ 601us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:22.674795 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:22.676247 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:22.677846 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:22.678815 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:22.679060 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:22.685869 (+ 6809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:22.685885 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:22.685918 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:22.685995 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:22.691004 (+ 5009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:22.692275 (+ 1271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:22.692284 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:22.692292 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:22.692644 (+ 352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:22.693214 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:22.693217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:22.693220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:22.693947 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:22.694211 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:22.694831 (+ 620us) server_negotiation.cc:299] Negotiation successful
0806 19:40:22.695394 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":439,"threads_started":1}
I20260806 19:40:22.702581 28169 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8377841e1e6541399143bd371d22d852" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39071 } }
I20260806 19:40:22.703096 28169 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:22.703243 28169 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8377841e1e6541399143bd371d22d852, State: Initialized, Role: FOLLOWER
I20260806 19:40:22.703925 28169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8377841e1e6541399143bd371d22d852" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39071 } }
I20260806 19:40:22.704504 28169 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:40:22.705247 28169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:40:22.705816 28169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:22.709750 28169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8377841e1e6541399143bd371d22d852" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39071 } }
I20260806 19:40:22.710744 28169 leader_election.cc:302] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8377841e1e6541399143bd371d22d852; no voters:
I20260806 19:40:22.711484 28169 leader_election.cc:288] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:40:22.711678 28174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:22.713133 28174 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [term 1 LEADER]: Becoming Leader. State: Replica: 8377841e1e6541399143bd371d22d852, State: Running, Role: LEADER
I20260806 19:40:22.714188 28174 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8377841e1e6541399143bd371d22d852" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39071 } }
I20260806 19:40:22.715369 28169 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:40:22.718020 28175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8377841e1e6541399143bd371d22d852" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8377841e1e6541399143bd371d22d852" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39071 } } }
I20260806 19:40:22.718612 28175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:22.718106 28176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8377841e1e6541399143bd371d22d852. Latest consensus state: current_term: 1 leader_uuid: "8377841e1e6541399143bd371d22d852" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8377841e1e6541399143bd371d22d852" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 39071 } } }
I20260806 19:40:22.718997 28176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852 [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:22.721976 28180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:40:22.728981 28180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:40:22.741748 28180 catalog_manager.cc:1347] Generated new cluster ID: a2d74d41060142fd82735c5f3b370329
I20260806 19:40:22.742027 28180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:40:22.760921 28180 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:40:22.763463 28180 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:40:22.778970 28180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852: Generated new TSK 0
I20260806 19:40:22.780283 28180 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:40:22.876504 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39071
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:38069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:23.324414 28195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:23.324780 28195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:23.324898 28195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:23.337209 28195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:23.337394 28195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:23.337682 28195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:40:23.350473 28195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:38069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:23.352273 28195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:23.354125 28195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:23.371361 28201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:23.371937 28203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:23.371989 28200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:23.373224 28195 server_base.cc:1034] running on GCE node
I20260806 19:40:23.374089 28195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:23.375315 28195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:23.376771 28195 hybrid_clock.cc:648] HybridClock initialized: now 1786045223376684 us; error 71 us; skew 500 ppm
I20260806 19:40:23.377229 28195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:23 dist-test-slave-ss4n krb5kdc[28087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045223, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:23.381422 28195 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:40:23.383572 28195 webserver.cc:460] Webserver started at http://127.10.25.1:34541/ using document root <none> and password file <none>
I20260806 19:40:23.384403 28195 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:23.384536 28195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:23.384881 28195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:23.389889 28195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "5175ff799e1649ad878e24bf2beaecde"
format_stamp: "Formatted at 2026-08-06 19:40:23 on dist-test-slave-ss4n"
server_key: "e28d48c80e8e4900325fd778f5278ded"
server_key_iv: "02ede8ab93c4655cbf685aa504adcb99"
server_key_version: "encryptionkey@0"
I20260806 19:40:23.391158 28195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "5175ff799e1649ad878e24bf2beaecde"
format_stamp: "Formatted at 2026-08-06 19:40:23 on dist-test-slave-ss4n"
server_key: "e28d48c80e8e4900325fd778f5278ded"
server_key_iv: "02ede8ab93c4655cbf685aa504adcb99"
server_key_version: "encryptionkey@0"
I20260806 19:40:23.399791 28195 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 19:40:23.404090 28210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:23.406203 28195 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:40:23.406718 28195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "5175ff799e1649ad878e24bf2beaecde"
format_stamp: "Formatted at 2026-08-06 19:40:23 on dist-test-slave-ss4n"
server_key: "e28d48c80e8e4900325fd778f5278ded"
server_key_iv: "02ede8ab93c4655cbf685aa504adcb99"
server_key_version: "encryptionkey@0"
I20260806 19:40:23.406996 28195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:23.433527 28195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:23.439267 28195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:23.439667 28195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:23.440927 28195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:23.442643 28195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:23.442734 28195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:23.442878 28195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:23.442945 28195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:23.492774 28195 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:36611
I20260806 19:40:23.492847 28323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:36611 every 8 connection(s)
I20260806 19:40:23.494391 28195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:40:23.498553 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 28195
I20260806 19:40:23.498857 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:40:23.499886 10340 external_mini_cluster.cc:1428] Setting key c8a762e224a4632a1875fd52df0da7c7
Aug 06 19:40:23 dist-test-slave-ss4n krb5kdc[28087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045223, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:40:23.523658 28326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:23.497642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:59219 (local address 127.10.25.62:39071)
0806 19:40:23.498181 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:23.498190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:23.499764 (+ 1574us) server_negotiation.cc:407] Connection header received
0806 19:40:23.502623 (+ 2859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:23.502629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:23.502723 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:23.502820 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:23.505676 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:23.506531 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:23.509832 (+ 3301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:23.510380 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:23.510775 (+ 395us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:23.516024 (+ 5249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:23.516032 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:23.516035 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:23.516080 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:23.519499 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:23.520526 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:23.520533 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:23.520535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:23.520683 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:23.521196 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:23.521201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:23.521203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:23.521583 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:23.521754 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:23.522771 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0806 19:40:23.523112 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":240,"threads_started":1}
I20260806 19:40:23.524637 28327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:23.498290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39071 (local address 127.10.25.1:59219)
0806 19:40:23.499486 (+ 1196us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:23.499566 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:23.502231 (+ 2665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:23.503207 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:23.503222 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:23.504243 (+ 1021us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:23.505326 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:23.505366 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:23.506787 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:23.506798 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:23.509525 (+ 2727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:23.509557 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:23.510765 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:23.510776 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:23.511183 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:23.512353 (+ 1170us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:40:23.512392 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:23.515689 (+ 3297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:23.519748 (+ 4059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:23.519760 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:23.519791 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:23.520349 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:23.520817 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:23.520823 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:23.520827 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:23.521052 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:23.521725 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:23.521735 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:23.522256 (+ 521us) client_negotiation.cc:769] Sending connection context
0806 19:40:23.522576 (+ 320us) client_negotiation.cc:240] Negotiation successful
0806 19:40:23.523142 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":551,"threads_started":1}
I20260806 19:40:23.526981 28324 heartbeater.cc:347] Connected to a master server at 127.10.25.62:39071
I20260806 19:40:23.527379 28324 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:23.528286 28324 heartbeater.cc:511] Master 127.10.25.62:39071 requested a full tablet report, sending...
I20260806 19:40:23.531301 28132 ts_manager.cc:194] Registered new tserver with Master: 5175ff799e1649ad878e24bf2beaecde (127.10.25.1:36611)
I20260806 19:40:23.533769 28132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:59219
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:40:23.582633 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39071
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:38069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:24.006580 28330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:24.006939 28330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:24.007043 28330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:24.016211 28330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:24.016423 28330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:24.016652 28330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:40:24.029758 28330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:38069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:24.031280 28330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:24.032729 28330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:24.048534 28336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:24.048552 28338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:24.048848 28335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:24.050658 28330 server_base.cc:1034] running on GCE node
I20260806 19:40:24.051524 28330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:24.052731 28330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:24.056295 28330 hybrid_clock.cc:648] HybridClock initialized: now 1786045224056174 us; error 80 us; skew 500 ppm
I20260806 19:40:24.056887 28330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:24 dist-test-slave-ss4n krb5kdc[28087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045224, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:24.061100 28330 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:40:24.063321 28330 webserver.cc:460] Webserver started at http://127.10.25.2:35509/ using document root <none> and password file <none>
I20260806 19:40:24.064297 28330 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:24.064492 28330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:24.064857 28330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:24.068607 28330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "aec7f190f4b34092abc97b6658f374f1"
format_stamp: "Formatted at 2026-08-06 19:40:24 on dist-test-slave-ss4n"
server_key: "3528b24849845322ccf4638ecda1d84a"
server_key_iv: "7412deea609f168ba454a8ba24dd175d"
server_key_version: "encryptionkey@0"
I20260806 19:40:24.069717 28330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "aec7f190f4b34092abc97b6658f374f1"
format_stamp: "Formatted at 2026-08-06 19:40:24 on dist-test-slave-ss4n"
server_key: "3528b24849845322ccf4638ecda1d84a"
server_key_iv: "7412deea609f168ba454a8ba24dd175d"
server_key_version: "encryptionkey@0"
I20260806 19:40:24.077049 28330 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 19:40:24.086311 28345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:24.091516 28330 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.005s sys 0.003s
I20260806 19:40:24.091827 28330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "aec7f190f4b34092abc97b6658f374f1"
format_stamp: "Formatted at 2026-08-06 19:40:24 on dist-test-slave-ss4n"
server_key: "3528b24849845322ccf4638ecda1d84a"
server_key_iv: "7412deea609f168ba454a8ba24dd175d"
server_key_version: "encryptionkey@0"
I20260806 19:40:24.092106 28330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:24.111752 28330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:24.117703 28330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:24.118059 28330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:24.119247 28330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:24.121465 28330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:24.121673 28330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 19:40:24.121858 28330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:24.121940 28330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:24.167593 28330 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:35445
I20260806 19:40:24.167654 28458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:35445 every 8 connection(s)
I20260806 19:40:24.169091 28330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:40:24.171633 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 28330
I20260806 19:40:24.171854 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:40:24.172554 10340 external_mini_cluster.cc:1428] Setting key 1f02986263ae7908e6de49a4e78bf260
Aug 06 19:40:24 dist-test-slave-ss4n krb5kdc[28087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045224, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:40:24.200109 28461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:24.172676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:52449 (local address 127.10.25.62:39071)
0806 19:40:24.173303 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:24.173309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:24.174348 (+ 1039us) server_negotiation.cc:407] Connection header received
0806 19:40:24.176672 (+ 2324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:24.176678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:24.176772 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:24.176912 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:24.178794 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.179570 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.181672 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.182624 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.182810 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:24.188454 (+ 5644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:24.188461 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:24.188464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:24.188516 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:24.194250 (+ 5734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:24.195469 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:24.195480 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:24.195487 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:24.195699 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:24.196320 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:24.196329 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:24.196337 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:24.197583 (+ 1246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:24.197777 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:24.199225 (+ 1448us) server_negotiation.cc:299] Negotiation successful
0806 19:40:24.199564 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":362,"threads_started":1}
I20260806 19:40:24.201757 28462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:24.173086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39071 (local address 127.10.25.2:52449)
0806 19:40:24.174103 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:24.174170 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:24.176342 (+ 2172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:24.177204 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:24.177214 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:24.177905 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:24.178559 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:24.178582 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.179925 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.179939 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:24.181401 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:24.181420 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.182924 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.182929 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:24.183173 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:24.184424 (+ 1251us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:40:24.184570 (+ 146us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:24.188039 (+ 3469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:24.194544 (+ 6505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:24.194565 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:24.194583 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:24.195246 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:24.195880 (+ 634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:24.195883 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:24.195886 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:24.196147 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:24.197931 (+ 1784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:24.197942 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:24.198641 (+ 699us) client_negotiation.cc:769] Sending connection context
0806 19:40:24.199097 (+ 456us) client_negotiation.cc:240] Negotiation successful
0806 19:40:24.199810 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":421,"threads_started":1}
I20260806 19:40:24.203644 28459 heartbeater.cc:347] Connected to a master server at 127.10.25.62:39071
I20260806 19:40:24.204057 28459 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:24.205623 28459 heartbeater.cc:511] Master 127.10.25.62:39071 requested a full tablet report, sending...
I20260806 19:40:24.207934 28132 ts_manager.cc:194] Registered new tserver with Master: aec7f190f4b34092abc97b6658f374f1 (127.10.25.2:35445)
I20260806 19:40:24.209035 28132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:52449
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:40:24.259235 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39071
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:38069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:40:24.538044 28324 heartbeater.cc:503] Master 127.10.25.62:39071 was elected leader, sending a full tablet report...
W20260806 19:40:24.681968 28465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:24.682328 28465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:24.682528 28465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:24.691290 28465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:24.691435 28465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:24.691659 28465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:40:24.705349 28465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:38069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:39071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:24.707020 28465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:24.708552 28465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:24.725747 28473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:24.725627 28471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:24.725649 28470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:24.728123 28465 server_base.cc:1034] running on GCE node
I20260806 19:40:24.729781 28465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:24.731135 28465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:24.732422 28465 hybrid_clock.cc:648] HybridClock initialized: now 1786045224732358 us; error 62 us; skew 500 ppm
I20260806 19:40:24.732844 28465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:24 dist-test-slave-ss4n krb5kdc[28087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045224, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:24.737367 28465 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:40:24.740070 28465 webserver.cc:460] Webserver started at http://127.10.25.3:38935/ using document root <none> and password file <none>
I20260806 19:40:24.741109 28465 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:24.741290 28465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:24.741756 28465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:24.745455 28465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "280ed8273b5e4c51a6482a2fe5ea3cd7"
format_stamp: "Formatted at 2026-08-06 19:40:24 on dist-test-slave-ss4n"
server_key: "716837969aea0cedd9fef03166b9b5e3"
server_key_iv: "5986449f3c783c59288412b5bbaff8aa"
server_key_version: "encryptionkey@0"
I20260806 19:40:24.746560 28465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "280ed8273b5e4c51a6482a2fe5ea3cd7"
format_stamp: "Formatted at 2026-08-06 19:40:24 on dist-test-slave-ss4n"
server_key: "716837969aea0cedd9fef03166b9b5e3"
server_key_iv: "5986449f3c783c59288412b5bbaff8aa"
server_key_version: "encryptionkey@0"
I20260806 19:40:24.753397 28465 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 19:40:24.758606 28480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:24.760582 28465 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 19:40:24.760844 28465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "280ed8273b5e4c51a6482a2fe5ea3cd7"
format_stamp: "Formatted at 2026-08-06 19:40:24 on dist-test-slave-ss4n"
server_key: "716837969aea0cedd9fef03166b9b5e3"
server_key_iv: "5986449f3c783c59288412b5bbaff8aa"
server_key_version: "encryptionkey@0"
I20260806 19:40:24.761096 28465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:24.784421 28465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:24.791281 28465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:24.791680 28465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:24.793435 28465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:24.795928 28465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:24.796109 28465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:24.796550 28465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:24.796653 28465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:24.847250 28465 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:35659
I20260806 19:40:24.847378 28593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:35659 every 8 connection(s)
I20260806 19:40:24.848706 28465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:40:24.858712 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 28465
I20260806 19:40:24.858922 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:40:24.859477 10340 external_mini_cluster.cc:1428] Setting key 5b421dbcb0c026c7f3d4da1b4c939fc9
Aug 06 19:40:24 dist-test-slave-ss4n krb5kdc[28087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045224, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:40:24.876252 28596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:24.852081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:52513 (local address 127.10.25.62:39071)
0806 19:40:24.852600 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:24.852605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:24.853903 (+ 1298us) server_negotiation.cc:407] Connection header received
0806 19:40:24.855975 (+ 2072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:24.855981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:24.856095 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:24.856227 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:24.858083 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.859273 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.861656 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.862361 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.862480 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:24.867356 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:24.867363 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:24.867367 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:24.867412 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:24.870732 (+ 3320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:24.872305 (+ 1573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:24.872312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:24.872314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:24.872492 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:24.873259 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:24.873263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:24.873266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:24.873756 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:24.873958 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:24.875474 (+ 1516us) server_negotiation.cc:299] Negotiation successful
0806 19:40:24.875825 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":279,"threads_started":1}
I20260806 19:40:24.877184 28597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:24.852743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:39071 (local address 127.10.25.3:52513)
0806 19:40:24.853721 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:24.853764 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:24.855621 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:24.856508 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:24.856518 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:24.857123 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:24.857842 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:24.857863 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.859490 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.859495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:24.861400 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:24.861418 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.862704 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.862715 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:24.863065 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:24.864019 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:40:24.864052 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:24.867010 (+ 2958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:24.871082 (+ 4072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:24.871096 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:24.871120 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:24.872036 (+ 916us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:24.872711 (+ 675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:24.872717 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:24.872720 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:24.873052 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:24.874002 (+ 950us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:24.874011 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:24.874853 (+ 842us) client_negotiation.cc:769] Sending connection context
0806 19:40:24.875261 (+ 408us) client_negotiation.cc:240] Negotiation successful
0806 19:40:24.875861 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":382,"threads_started":1}
I20260806 19:40:24.878789 28594 heartbeater.cc:347] Connected to a master server at 127.10.25.62:39071
I20260806 19:40:24.879231 28594 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:24.880101 28594 heartbeater.cc:511] Master 127.10.25.62:39071 requested a full tablet report, sending...
I20260806 19:40:24.881947 28132 ts_manager.cc:194] Registered new tserver with Master: 280ed8273b5e4c51a6482a2fe5ea3cd7 (127.10.25.3:35659)
I20260806 19:40:24.882843 28132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:52513
I20260806 19:40:24.890519 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:40:24.916733 28596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:24.896964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41246 (local address 127.10.25.62:39071)
0806 19:40:24.897214 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:24.897221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:24.897413 (+ 192us) server_negotiation.cc:407] Connection header received
0806 19:40:24.897752 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:24.897756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:24.897861 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:24.897973 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:24.900180 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.901087 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.903124 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.903576 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.903739 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:24.907402 (+ 3663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:24.907410 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:24.907455 (+ 45us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:24.907508 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:24.911734 (+ 4226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:24.913103 (+ 1369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:24.913109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:24.913112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:24.913361 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:24.914006 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:24.914011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:24.914017 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:24.914680 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:24.915141 (+ 461us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:24.915787 (+ 646us) server_negotiation.cc:299] Negotiation successful
0806 19:40:24.916090 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 19:40:24.923336 28132 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41246:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:40:24.932013 28132 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:40:24.965911 28610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:24.951498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:36611 (local address 127.0.0.1:35806)
0806 19:40:24.953554 (+ 2056us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:24.953578 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:24.953742 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:24.955511 (+ 1769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:24.955520 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:24.955654 (+ 134us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:24.956888 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:24.956903 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.959431 (+ 2528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.959440 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:24.962060 (+ 2620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:24.962078 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.964287 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.964294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:24.964751 (+ 457us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:24.964990 (+ 239us) client_negotiation.cc:769] Sending connection context
0806 19:40:24.965198 (+ 208us) client_negotiation.cc:240] Negotiation successful
0806 19:40:24.965461 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1881,"thread_start_us":1932,"threads_started":1}
I20260806 19:40:24.966130 28609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:24.951424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56224 (local address 127.10.25.3:35659)
0806 19:40:24.954543 (+ 3119us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:24.954562 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:24.954614 (+ 52us) server_negotiation.cc:407] Connection header received
0806 19:40:24.954764 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:24.954778 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:24.955109 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:24.955841 (+ 732us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:24.957141 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.958703 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.962844 (+ 4141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.964321 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.964449 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:24.964697 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:24.965343 (+ 646us) server_negotiation.cc:299] Negotiation successful
0806 19:40:24.965616 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2848,"thread_start_us":2605,"threads_started":1}
I20260806 19:40:24.966650 28611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:24.952268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35806 (local address 127.10.25.1:36611)
0806 19:40:24.953575 (+ 1307us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:24.953587 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:24.954310 (+ 723us) server_negotiation.cc:407] Connection header received
0806 19:40:24.954442 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:24.954451 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:24.954885 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:24.955157 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:24.957605 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.959256 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.962339 (+ 3083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.964048 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.964209 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:24.964581 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:24.966068 (+ 1487us) server_negotiation.cc:299] Negotiation successful
0806 19:40:24.966293 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1065,"thread_start_us":903,"threads_started":1}
I20260806 19:40:24.967222 28608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:24.951239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:35659 (local address 127.0.0.1:56224)
0806 19:40:24.952468 (+ 1229us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:24.952540 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:24.952794 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:24.955503 (+ 2709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:24.955519 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:24.955572 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:24.956231 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:24.956251 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.958871 (+ 2620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.958878 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:24.962623 (+ 3745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:24.962640 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.964781 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.964786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:24.964979 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:24.965009 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 19:40:24.965267 (+ 258us) client_negotiation.cc:240] Negotiation successful
0806 19:40:24.965545 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"mutex_wait_us":184,"thread_start_us":659,"threads_started":1}
I20260806 19:40:24.967825 28612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:24.953888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:35445 (local address 127.0.0.1:48102)
0806 19:40:24.957599 (+ 3711us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:24.957622 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:24.957865 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:24.958912 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:24.958918 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:24.958952 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:24.959471 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:24.959484 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.961629 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.961642 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:24.963562 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:24.963577 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.966930 (+ 3353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.966936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:24.967087 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:24.967113 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 19:40:24.967253 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 19:40:24.967407 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3552,"mutex_wait_us":4,"thread_start_us":3358,"threads_started":1}
I20260806 19:40:24.967849 28613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:24.954366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48102 (local address 127.10.25.2:35445)
0806 19:40:24.956933 (+ 2567us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:24.956945 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:24.957737 (+ 792us) server_negotiation.cc:407] Connection header received
0806 19:40:24.958107 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:24.958117 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:24.958411 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:24.958616 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:24.959664 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.961120 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.963791 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:24.965745 (+ 1954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:24.966506 (+ 761us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:24.966749 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:24.967318 (+ 569us) server_negotiation.cc:299] Negotiation successful
0806 19:40:24.967513 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2343,"thread_start_us":2159,"threads_started":1}
I20260806 19:40:24.971951 28529 tablet_service.cc:1467] Processing CreateTablet for tablet e6eb743fd64f4029b2e3d07d8fff7d42 (DEFAULT_TABLE table=test-table [id=e2c192ee4d9d48b8b717812c7dcb0ade]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:24.971951 28259 tablet_service.cc:1467] Processing CreateTablet for tablet e6eb743fd64f4029b2e3d07d8fff7d42 (DEFAULT_TABLE table=test-table [id=e2c192ee4d9d48b8b717812c7dcb0ade]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:24.971940 28394 tablet_service.cc:1467] Processing CreateTablet for tablet e6eb743fd64f4029b2e3d07d8fff7d42 (DEFAULT_TABLE table=test-table [id=e2c192ee4d9d48b8b717812c7dcb0ade]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:24.973748 28529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6eb743fd64f4029b2e3d07d8fff7d42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:24.973748 28259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6eb743fd64f4029b2e3d07d8fff7d42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:24.973742 28394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6eb743fd64f4029b2e3d07d8fff7d42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:24.989204 28614 tablet_bootstrap.cc:492] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1: Bootstrap starting.
I20260806 19:40:24.991852 28615 tablet_bootstrap.cc:492] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde: Bootstrap starting.
I20260806 19:40:24.994030 28616 tablet_bootstrap.cc:492] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7: Bootstrap starting.
I20260806 19:40:24.994371 28614 tablet_bootstrap.cc:654] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:24.996030 28614 log.cc:824] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:24.998147 28615 tablet_bootstrap.cc:654] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:24.999579 28615 log.cc:824] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:24.999749 28616 tablet_bootstrap.cc:654] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:25.001242 28616 log.cc:824] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:25.001653 28614 tablet_bootstrap.cc:492] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1: No bootstrap required, opened a new log
I20260806 19:40:25.002038 28614 ts_tablet_manager.cc:1397] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.003s
I20260806 19:40:25.003285 28615 tablet_bootstrap.cc:492] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde: No bootstrap required, opened a new log
I20260806 19:40:25.003620 28615 ts_tablet_manager.cc:1397] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.001s
I20260806 19:40:25.004994 28616 tablet_bootstrap.cc:492] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7: No bootstrap required, opened a new log
I20260806 19:40:25.005304 28616 ts_tablet_manager.cc:1397] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260806 19:40:25.013700 28614 raft_consensus.cc:348] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed8273b5e4c51a6482a2fe5ea3cd7" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 35659 } } peers { permanent_uuid: "5175ff799e1649ad878e24bf2beaecde" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 36611 } } peers { permanent_uuid: "aec7f190f4b34092abc97b6658f374f1" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35445 } }
I20260806 19:40:25.014591 28614 raft_consensus.cc:374] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:25.014847 28614 raft_consensus.cc:729] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aec7f190f4b34092abc97b6658f374f1, State: Initialized, Role: FOLLOWER
I20260806 19:40:25.015820 28614 consensus_queue.cc:260] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed8273b5e4c51a6482a2fe5ea3cd7" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 35659 } } peers { permanent_uuid: "5175ff799e1649ad878e24bf2beaecde" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 36611 } } peers { permanent_uuid: "aec7f190f4b34092abc97b6658f374f1" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35445 } }
I20260806 19:40:25.016985 28616 raft_consensus.cc:348] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed8273b5e4c51a6482a2fe5ea3cd7" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 35659 } } peers { permanent_uuid: "5175ff799e1649ad878e24bf2beaecde" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 36611 } } peers { permanent_uuid: "aec7f190f4b34092abc97b6658f374f1" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35445 } }
I20260806 19:40:25.016860 28615 raft_consensus.cc:348] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed8273b5e4c51a6482a2fe5ea3cd7" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 35659 } } peers { permanent_uuid: "5175ff799e1649ad878e24bf2beaecde" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 36611 } } peers { permanent_uuid: "aec7f190f4b34092abc97b6658f374f1" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35445 } }
I20260806 19:40:25.017738 28616 raft_consensus.cc:374] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:25.017867 28616 raft_consensus.cc:729] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 280ed8273b5e4c51a6482a2fe5ea3cd7, State: Initialized, Role: FOLLOWER
I20260806 19:40:25.017923 28615 raft_consensus.cc:374] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:25.018105 28615 raft_consensus.cc:729] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5175ff799e1649ad878e24bf2beaecde, State: Initialized, Role: FOLLOWER
I20260806 19:40:25.018813 28616 consensus_queue.cc:260] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed8273b5e4c51a6482a2fe5ea3cd7" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 35659 } } peers { permanent_uuid: "5175ff799e1649ad878e24bf2beaecde" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 36611 } } peers { permanent_uuid: "aec7f190f4b34092abc97b6658f374f1" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35445 } }
I20260806 19:40:25.019277 28615 consensus_queue.cc:260] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed8273b5e4c51a6482a2fe5ea3cd7" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 35659 } } peers { permanent_uuid: "5175ff799e1649ad878e24bf2beaecde" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 36611 } } peers { permanent_uuid: "aec7f190f4b34092abc97b6658f374f1" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35445 } }
I20260806 19:40:25.020759 28594 heartbeater.cc:503] Master 127.10.25.62:39071 was elected leader, sending a full tablet report...
I20260806 19:40:25.022287 28616 ts_tablet_manager.cc:1428] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7: Time spent starting tablet: real 0.017s user 0.014s sys 0.004s
I20260806 19:40:25.023972 28459 heartbeater.cc:503] Master 127.10.25.62:39071 was elected leader, sending a full tablet report...
I20260806 19:40:25.026679 28614 ts_tablet_manager.cc:1428] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1: Time spent starting tablet: real 0.025s user 0.017s sys 0.004s
I20260806 19:40:25.029696 28615 ts_tablet_manager.cc:1428] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde: Time spent starting tablet: real 0.026s user 0.022s sys 0.004s
W20260806 19:40:25.101162 28595 tablet.cc:2369] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:40:25.171655 28620 raft_consensus.cc:482] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260806 19:40:25.172055 28460 tablet.cc:2369] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:40:25.172263 28620 raft_consensus.cc:504] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed8273b5e4c51a6482a2fe5ea3cd7" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 35659 } } peers { permanent_uuid: "5175ff799e1649ad878e24bf2beaecde" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 36611 } } peers { permanent_uuid: "aec7f190f4b34092abc97b6658f374f1" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35445 } }
I20260806 19:40:25.175180 28620 leader_election.cc:288] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 280ed8273b5e4c51a6482a2fe5ea3cd7 (127.10.25.3:35659), 5175ff799e1649ad878e24bf2beaecde (127.10.25.1:36611)
I20260806 19:40:25.186508 28609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:25.175102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:55795 (local address 127.10.25.3:35659)
0806 19:40:25.175323 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:25.175329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:25.178629 (+ 3300us) server_negotiation.cc:407] Connection header received
0806 19:40:25.178876 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:25.178882 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:25.178989 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:25.179100 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:25.179956 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:25.180881 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:25.184244 (+ 3363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:25.185230 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:25.185451 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:25.185624 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:25.186039 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 19:40:25.186193 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 19:40:25.186532 28611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:25.175303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:40283 (local address 127.10.25.1:36611)
0806 19:40:25.175566 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:25.175576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:25.178681 (+ 3105us) server_negotiation.cc:407] Connection header received
0806 19:40:25.179008 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:25.179018 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:25.179118 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:25.179231 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:25.180185 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:25.181315 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:25.184259 (+ 2944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:25.185200 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:25.185410 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:25.185586 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:25.186029 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 19:40:25.186205 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 19:40:25.186829 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:25.175752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:36611 (local address 127.10.25.2:40283)
0806 19:40:25.178549 (+ 2797us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:25.178568 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:25.178787 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:25.179332 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:25.179337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:25.179368 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:25.179957 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:25.179970 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:25.181511 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:25.181518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:25.183975 (+ 2457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:25.183984 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:25.185543 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:25.185553 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:25.185768 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:25.185827 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 19:40:25.185995 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 19:40:25.186205 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2668,"mutex_wait_us":91,"thread_start_us":2497,"threads_started":1}
I20260806 19:40:25.187148 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:25.175603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:35659 (local address 127.10.25.2:55795)
0806 19:40:25.178421 (+ 2818us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:25.178455 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:25.178692 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:25.179145 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:25.179149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:25.179176 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:25.179734 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:25.179753 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:25.181070 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:25.181077 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:25.183937 (+ 2860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:25.183964 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:25.185629 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:25.185639 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:25.185828 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:25.185857 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 19:40:25.185994 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 19:40:25.186190 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2513,"mutex_wait_us":71,"thread_start_us":1951,"threads_started":1}
I20260806 19:40:25.188215 28279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6eb743fd64f4029b2e3d07d8fff7d42" candidate_uuid: "aec7f190f4b34092abc97b6658f374f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5175ff799e1649ad878e24bf2beaecde" is_pre_election: true
I20260806 19:40:25.188336 28549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6eb743fd64f4029b2e3d07d8fff7d42" candidate_uuid: "aec7f190f4b34092abc97b6658f374f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280ed8273b5e4c51a6482a2fe5ea3cd7" is_pre_election: true
I20260806 19:40:25.188956 28279 raft_consensus.cc:2454] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aec7f190f4b34092abc97b6658f374f1 in term 0.
I20260806 19:40:25.188968 28549 raft_consensus.cc:2454] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aec7f190f4b34092abc97b6658f374f1 in term 0.
I20260806 19:40:25.190119 28347 leader_election.cc:302] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5175ff799e1649ad878e24bf2beaecde, aec7f190f4b34092abc97b6658f374f1; no voters:
I20260806 19:40:25.190757 28620 raft_consensus.cc:2781] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:40:25.190963 28620 raft_consensus.cc:482] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:40:25.191162 28620 raft_consensus.cc:3037] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:25.193851 28620 raft_consensus.cc:504] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed8273b5e4c51a6482a2fe5ea3cd7" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 35659 } } peers { permanent_uuid: "5175ff799e1649ad878e24bf2beaecde" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 36611 } } peers { permanent_uuid: "aec7f190f4b34092abc97b6658f374f1" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35445 } }
I20260806 19:40:25.194547 28620 leader_election.cc:288] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [CANDIDATE]: Term 1 election: Requested vote from peers 280ed8273b5e4c51a6482a2fe5ea3cd7 (127.10.25.3:35659), 5175ff799e1649ad878e24bf2beaecde (127.10.25.1:36611)
I20260806 19:40:25.195282 28549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6eb743fd64f4029b2e3d07d8fff7d42" candidate_uuid: "aec7f190f4b34092abc97b6658f374f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280ed8273b5e4c51a6482a2fe5ea3cd7"
I20260806 19:40:25.195297 28279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6eb743fd64f4029b2e3d07d8fff7d42" candidate_uuid: "aec7f190f4b34092abc97b6658f374f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5175ff799e1649ad878e24bf2beaecde"
I20260806 19:40:25.195544 28549 raft_consensus.cc:3037] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:25.195617 28279 raft_consensus.cc:3037] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:25.198231 28549 raft_consensus.cc:2454] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aec7f190f4b34092abc97b6658f374f1 in term 1.
I20260806 19:40:25.198854 28279 raft_consensus.cc:2454] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aec7f190f4b34092abc97b6658f374f1 in term 1.
I20260806 19:40:25.199163 28348 leader_election.cc:302] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 280ed8273b5e4c51a6482a2fe5ea3cd7, aec7f190f4b34092abc97b6658f374f1; no voters:
I20260806 19:40:25.199612 28620 raft_consensus.cc:2781] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:25.200206 28620 raft_consensus.cc:686] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [term 1 LEADER]: Becoming Leader. State: Replica: aec7f190f4b34092abc97b6658f374f1, State: Running, Role: LEADER
I20260806 19:40:25.200738 28620 consensus_queue.cc:237] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed8273b5e4c51a6482a2fe5ea3cd7" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 35659 } } peers { permanent_uuid: "5175ff799e1649ad878e24bf2beaecde" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 36611 } } peers { permanent_uuid: "aec7f190f4b34092abc97b6658f374f1" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35445 } }
I20260806 19:40:25.206804 28130 catalog_manager.cc:5495] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to aec7f190f4b34092abc97b6658f374f1 (127.10.25.2). New cstate: current_term: 1 leader_uuid: "aec7f190f4b34092abc97b6658f374f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed8273b5e4c51a6482a2fe5ea3cd7" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 35659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5175ff799e1649ad878e24bf2beaecde" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 36611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aec7f190f4b34092abc97b6658f374f1" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 35445 } health_report { overall_health: HEALTHY } } }
W20260806 19:40:25.247862 28325 tablet.cc:2369] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:40:25.280665 28613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:25.272939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48112 (local address 127.10.25.2:35445)
0806 19:40:25.273224 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:25.273230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:25.273254 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:40:25.273472 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:25.273476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:25.273653 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:25.273816 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:40:25.274310 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:25.275048 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:25.276950 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:25.277416 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:25.277742 (+ 326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:25.278761 (+ 1019us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:40:25.279643 (+ 882us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:40:25.279844 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:25.280093 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 19:40:25.280326 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260806 19:40:25.290489 28279 raft_consensus.cc:1261] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde [term 1 FOLLOWER]: Refusing update from remote peer aec7f190f4b34092abc97b6658f374f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:40:25.290488 28549 raft_consensus.cc:1261] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7 [term 1 FOLLOWER]: Refusing update from remote peer aec7f190f4b34092abc97b6658f374f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:40:25.292003 28625 consensus_queue.cc:1035] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "280ed8273b5e4c51a6482a2fe5ea3cd7" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 35659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:40:25.292721 28620 consensus_queue.cc:1035] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5175ff799e1649ad878e24bf2beaecde" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 36611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:40:25.307934 28628 mvcc.cc:205] Tried to move back new op lower bound from 7315641242777202688 to 7315641242428346368. Current Snapshot: MvccSnapshot[applied={T|T < 7315641242777202688}]
I20260806 19:40:25.326804 28130 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:40:25.327172 28130 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:40:25.331822 28130 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8377841e1e6541399143bd371d22d852: Sending DeleteTablet for 3 replicas of tablet e6eb743fd64f4029b2e3d07d8fff7d42
I20260806 19:40:25.333233 28529 tablet_service.cc:1514] Processing DeleteTablet for tablet e6eb743fd64f4029b2e3d07d8fff7d42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:25 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:56224
I20260806 19:40:25.333722 28394 tablet_service.cc:1514] Processing DeleteTablet for tablet e6eb743fd64f4029b2e3d07d8fff7d42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:25 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:48102
I20260806 19:40:25.334100 28259 tablet_service.cc:1514] Processing DeleteTablet for tablet e6eb743fd64f4029b2e3d07d8fff7d42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:25 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:35806
I20260806 19:40:25.335008 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 28195
I20260806 19:40:25.349195 28639 tablet_replica.cc:330] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1: stopping tablet replica
I20260806 19:40:25.349974 28639 raft_consensus.cc:2229] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:40:25.351051 28639 raft_consensus.cc:2258] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:25.353029 28638 tablet_replica.cc:330] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7: stopping tablet replica
I20260806 19:40:25.353713 28638 raft_consensus.cc:2229] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:40:25.354521 28638 raft_consensus.cc:2258] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:25.355506 28640 tablet_replica.cc:330] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde: stopping tablet replica
I20260806 19:40:25.356004 28640 raft_consensus.cc:2229] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:40:25.356860 28639 ts_tablet_manager.cc:1905] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:25.357057 28640 raft_consensus.cc:2258] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:25.360966 28640 ts_tablet_manager.cc:1905] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:25.367836 28639 ts_tablet_manager.cc:1918] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:40:25.368096 28639 log.cc:1196] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/e6eb743fd64f4029b2e3d07d8fff7d42
I20260806 19:40:25.368844 28638 ts_tablet_manager.cc:1905] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:25.369437 28639 ts_tablet_manager.cc:1939] T e6eb743fd64f4029b2e3d07d8fff7d42 P aec7f190f4b34092abc97b6658f374f1: Deleting consensus metadata
I20260806 19:40:25.371315 28640 ts_tablet_manager.cc:1918] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:40:25.371519 28640 log.cc:1196] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/e6eb743fd64f4029b2e3d07d8fff7d42
I20260806 19:40:25.371896 28117 catalog_manager.cc:4841] TS aec7f190f4b34092abc97b6658f374f1 (127.10.25.2:35445): tablet e6eb743fd64f4029b2e3d07d8fff7d42 (table test-table [id=e2c192ee4d9d48b8b717812c7dcb0ade]) successfully deleted
I20260806 19:40:25.372097 28640 ts_tablet_manager.cc:1939] T e6eb743fd64f4029b2e3d07d8fff7d42 P 5175ff799e1649ad878e24bf2beaecde: Deleting consensus metadata
I20260806 19:40:25.374445 28117 catalog_manager.cc:4841] TS 5175ff799e1649ad878e24bf2beaecde (127.10.25.1:36611): tablet e6eb743fd64f4029b2e3d07d8fff7d42 (table test-table [id=e2c192ee4d9d48b8b717812c7dcb0ade]) successfully deleted
Aug 06 19:40:25 dist-test-slave-ss4n krb5kdc[28087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:40:25.377796 28638 ts_tablet_manager.cc:1918] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:40:25.378049 28638 log.cc:1196] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/e6eb743fd64f4029b2e3d07d8fff7d42
I20260806 19:40:25.378578 28638 ts_tablet_manager.cc:1939] T e6eb743fd64f4029b2e3d07d8fff7d42 P 280ed8273b5e4c51a6482a2fe5ea3cd7: Deleting consensus metadata
I20260806 19:40:25.380741 28118 catalog_manager.cc:4841] TS 280ed8273b5e4c51a6482a2fe5ea3cd7 (127.10.25.3:35659): tablet e6eb743fd64f4029b2e3d07d8fff7d42 (table test-table [id=e2c192ee4d9d48b8b717812c7dcb0ade]) successfully deleted
I20260806 19:40:25.387588 28611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:25.336382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35812 (local address 127.10.25.1:36611)
0806 19:40:25.336594 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:25.336605 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:25.353306 (+ 16701us) server_negotiation.cc:407] Connection header received
0806 19:40:25.355197 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:25.355207 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:25.355622 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:25.355938 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:25.358518 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:25.359836 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:25.363086 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:25.363848 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:25.363995 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:25.377249 (+ 13254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:25.377272 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:25.377296 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:25.377360 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:25.382954 (+ 5594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:25.384070 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:25.384083 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:25.384087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:25.384308 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:25.384871 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:25.384879 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:25.384883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:25.385533 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:25.385911 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:25.386799 (+ 888us) server_negotiation.cc:299] Negotiation successful
0806 19:40:25.387108 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 19:40:25.388440 28319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:25.502291 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 28195
I20260806 19:40:25.528017 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 28330
Aug 06 19:40:25 dist-test-slave-ss4n krb5kdc[28087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:40:25.556411 28613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:25.528994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48118 (local address 127.10.25.2:35445)
0806 19:40:25.529245 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:25.529252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:25.529281 (+ 29us) server_negotiation.cc:407] Connection header received
0806 19:40:25.529451 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:25.529458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:25.529587 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:25.529746 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:25.531179 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:25.532725 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:25.534518 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:25.535273 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:25.535408 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:25.545733 (+ 10325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:25.545761 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:25.545785 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:25.545849 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:25.551461 (+ 5612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:25.552494 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:25.552514 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:25.552518 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:25.552753 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:25.553727 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:25.553734 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:25.553738 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:25.554449 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:25.554860 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:25.555267 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 19:40:25.555866 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 19:40:25.557780 28454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:25.688367 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 28330
I20260806 19:40:25.715050 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 28465
Aug 06 19:40:25 dist-test-slave-ss4n krb5kdc[28087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:40:25.738361 28644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:25.716318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56228 (local address 127.10.25.3:35659)
0806 19:40:25.716910 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:25.716914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:25.716931 (+ 17us) server_negotiation.cc:407] Connection header received
0806 19:40:25.716993 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:25.716997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:25.717057 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:25.717167 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:25.719261 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:25.720072 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:25.722149 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:25.722622 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:25.722742 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:25.729229 (+ 6487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:25.729247 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:25.729264 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:25.729311 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:25.733343 (+ 4032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:25.734925 (+ 1582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:25.734935 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:25.734937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:25.735145 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:25.735919 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:25.736024 (+ 105us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:25.736027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:25.736667 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:25.737015 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:25.737764 (+ 749us) server_negotiation.cc:299] Negotiation successful
0806 19:40:25.738015 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":364,"threads_started":1}
I20260806 19:40:25.739152 28589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:25.841603 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 28465
I20260806 19:40:25.864172 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 28100
I20260806 19:40:25.866061 28164 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:25.995780 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 28100
2026-08-06T19:40:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5592 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T19:40:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:40:26Z Disabled control of system clock
I20260806 19:40:26.102422 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:35541
--webserver_interface=127.10.25.62
--webserver_port=0
--builtin_ntp_servers=127.10.25.20:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:35541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 19:40:26.515185 28650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:26.515937 28650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:26.516343 28650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:26.527731 28650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:40:26.528016 28650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:26.528128 28650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:26.528270 28650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:40:26.528364 28650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:40:26.541316 28650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:35541
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:35541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:26.543015 28650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:26.546247 28650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:26.562556 28655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:26.562577 28658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:26.562769 28656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:26.564417 28650 server_base.cc:1034] running on GCE node
I20260806 19:40:26.565435 28650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:26.567350 28650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:26.568694 28650 hybrid_clock.cc:648] HybridClock initialized: now 1786045226568600 us; error 72 us; skew 500 ppm
I20260806 19:40:26.569271 28650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:40:26.572295 28650 webserver.cc:460] Webserver started at http://127.10.25.62:46539/ using document root <none> and password file <none>
I20260806 19:40:26.573225 28650 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:26.573331 28650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:26.573802 28650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:26.577105 28650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "a08d0da32e404e0fbe21330dc4fd10ff"
format_stamp: "Formatted at 2026-08-06 19:40:26 on dist-test-slave-ss4n"
server_key: "27b651875b2be4b3f235aa90092d2d4f"
server_key_iv: "ad7583c0e5fe64c131bb3c6202ff1492"
server_key_version: "encryptionkey@0"
I20260806 19:40:26.578207 28650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "a08d0da32e404e0fbe21330dc4fd10ff"
format_stamp: "Formatted at 2026-08-06 19:40:26 on dist-test-slave-ss4n"
server_key: "27b651875b2be4b3f235aa90092d2d4f"
server_key_iv: "ad7583c0e5fe64c131bb3c6202ff1492"
server_key_version: "encryptionkey@0"
I20260806 19:40:26.585021 28650 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 19:40:26.590806 28664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:26.593353 28650 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260806 19:40:26.594020 28650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "a08d0da32e404e0fbe21330dc4fd10ff"
format_stamp: "Formatted at 2026-08-06 19:40:26 on dist-test-slave-ss4n"
server_key: "27b651875b2be4b3f235aa90092d2d4f"
server_key_iv: "ad7583c0e5fe64c131bb3c6202ff1492"
server_key_version: "encryptionkey@0"
I20260806 19:40:26.594434 28650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:26.627103 28650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:26.628225 28650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:26.628598 28650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:26.659251 28650 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:35541
I20260806 19:40:26.659559 28716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:35541 every 8 connection(s)
I20260806 19:40:26.661834 28650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:40:26.666723 28717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:26.671931 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 28650
I20260806 19:40:26.672241 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:40:26.672789 10340 external_mini_cluster.cc:1428] Setting key 0d9c7bad7101ce99d81f80ba23070765
I20260806 19:40:26.688999 28717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff: Bootstrap starting.
I20260806 19:40:26.694164 28717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:26.695283 28720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:26.676520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57300 (local address 127.10.25.62:35541)
0806 19:40:26.678073 (+ 1553us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:26.678094 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:26.678147 (+ 53us) server_negotiation.cc:407] Connection header received
0806 19:40:26.680094 (+ 1947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:26.680120 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:26.680649 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:26.681067 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:26.682641 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:26.685238 (+ 2597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:26.688123 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:26.689039 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:26.689433 (+ 394us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:26.691220 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:26.691232 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:26.691252 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:26.691966 (+ 714us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:26.692084 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:26.692114 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:26.692304 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:26.692620 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 19:40:26.693012 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1020,"thread_start_us":705,"threads_started":1}
I20260806 19:40:26.695871 28717 log.cc:824] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:26.701829 28717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff: No bootstrap required, opened a new log
I20260806 19:40:26.714529 28717 raft_consensus.cc:348] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08d0da32e404e0fbe21330dc4fd10ff" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35541 } }
I20260806 19:40:26.715180 28717 raft_consensus.cc:374] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:26.715369 28717 raft_consensus.cc:729] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a08d0da32e404e0fbe21330dc4fd10ff, State: Initialized, Role: FOLLOWER
I20260806 19:40:26.716666 28717 consensus_queue.cc:260] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08d0da32e404e0fbe21330dc4fd10ff" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35541 } }
I20260806 19:40:26.717382 28717 raft_consensus.cc:388] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:40:26.718160 28717 raft_consensus.cc:482] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:40:26.718971 28717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:26.723747 28717 raft_consensus.cc:504] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08d0da32e404e0fbe21330dc4fd10ff" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35541 } }
I20260806 19:40:26.724575 28717 leader_election.cc:302] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a08d0da32e404e0fbe21330dc4fd10ff; no voters:
I20260806 19:40:26.725418 28717 leader_election.cc:288] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:40:26.725929 28722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:26.729413 28722 raft_consensus.cc:686] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [term 1 LEADER]: Becoming Leader. State: Replica: a08d0da32e404e0fbe21330dc4fd10ff, State: Running, Role: LEADER
I20260806 19:40:26.731050 28717 sys_catalog.cc:564] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:40:26.730947 28722 consensus_queue.cc:237] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08d0da32e404e0fbe21330dc4fd10ff" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35541 } }
I20260806 19:40:26.735455 28724 sys_catalog.cc:455] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader a08d0da32e404e0fbe21330dc4fd10ff. Latest consensus state: current_term: 1 leader_uuid: "a08d0da32e404e0fbe21330dc4fd10ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08d0da32e404e0fbe21330dc4fd10ff" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35541 } } }
I20260806 19:40:26.736315 28724 sys_catalog.cc:458] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:26.735718 28723 sys_catalog.cc:455] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a08d0da32e404e0fbe21330dc4fd10ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a08d0da32e404e0fbe21330dc4fd10ff" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35541 } } }
I20260806 19:40:26.736683 28723 sys_catalog.cc:458] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:26.738276 28731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:40:26.756577 28731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:40:26.772346 28731 catalog_manager.cc:1347] Generated new cluster ID: 3ef06e58af0645fdb77bdf3100b15698
I20260806 19:40:26.772521 28731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:40:26.792585 28731 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:40:26.794842 28731 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:40:26.805925 28731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff: Generated new TSK 0
I20260806 19:40:26.808452 28731 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:40:26.828871 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:35541
--builtin_ntp_servers=127.10.25.20:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 19:40:27.284041 28741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:27.284392 28741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:27.284497 28741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:27.293949 28741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:27.294103 28741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:27.294323 28741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:40:27.305091 28741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:35541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:27.306720 28741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:27.309194 28741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:27.328190 28746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:27.328137 28747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:27.328842 28749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:27.329376 28741 server_base.cc:1034] running on GCE node
I20260806 19:40:27.330153 28741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:27.331471 28741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:27.332749 28741 hybrid_clock.cc:648] HybridClock initialized: now 1786045227332688 us; error 57 us; skew 500 ppm
I20260806 19:40:27.333230 28741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:40:27.336241 28741 webserver.cc:460] Webserver started at http://127.10.25.1:43479/ using document root <none> and password file <none>
I20260806 19:40:27.337141 28741 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:27.337277 28741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:27.337659 28741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:27.340953 28741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "d20d37e070334c93af2417638a2ac308"
format_stamp: "Formatted at 2026-08-06 19:40:27 on dist-test-slave-ss4n"
server_key: "03551891c6b3ac9703ee7d6cab0009a5"
server_key_iv: "50ce8270a9ead3052e5b7e8237f26f73"
server_key_version: "encryptionkey@0"
I20260806 19:40:27.341998 28741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "d20d37e070334c93af2417638a2ac308"
format_stamp: "Formatted at 2026-08-06 19:40:27 on dist-test-slave-ss4n"
server_key: "03551891c6b3ac9703ee7d6cab0009a5"
server_key_iv: "50ce8270a9ead3052e5b7e8237f26f73"
server_key_version: "encryptionkey@0"
I20260806 19:40:27.349373 28741 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260806 19:40:27.355017 28755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:27.357736 28741 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 19:40:27.358069 28741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "d20d37e070334c93af2417638a2ac308"
format_stamp: "Formatted at 2026-08-06 19:40:27 on dist-test-slave-ss4n"
server_key: "03551891c6b3ac9703ee7d6cab0009a5"
server_key_iv: "50ce8270a9ead3052e5b7e8237f26f73"
server_key_version: "encryptionkey@0"
I20260806 19:40:27.358615 28741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:27.376848 28741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:27.378059 28741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:27.378427 28741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:27.379654 28741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:27.381398 28741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:27.381575 28741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:27.381716 28741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:27.381815 28741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:27.434340 28741 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:43793
I20260806 19:40:27.434582 28870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:43793 every 8 connection(s)
I20260806 19:40:27.436049 28741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:40:27.442298 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 28741
I20260806 19:40:27.442519 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:40:27.443372 10340 external_mini_cluster.cc:1428] Setting key 297f32bbec9986bd29c45746812a238f
I20260806 19:40:27.447463 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:35541
--builtin_ntp_servers=127.10.25.20:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 19:40:27.458559 28873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:27.441138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:51911 (local address 127.10.25.62:35541)
0806 19:40:27.442108 (+ 970us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:27.442116 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:27.444321 (+ 2205us) server_negotiation.cc:407] Connection header received
0806 19:40:27.448002 (+ 3681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:27.448011 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:27.448137 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:27.448279 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:27.450041 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:27.451285 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:27.453516 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:27.454522 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:27.454715 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:27.456800 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:27.456810 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:27.456818 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:27.456851 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:27.456916 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:27.456938 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:27.457053 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:27.457873 (+ 820us) server_negotiation.cc:299] Negotiation successful
0806 19:40:27.458026 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":585,"threads_started":1}
I20260806 19:40:27.460427 28874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:27.442140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:35541 (local address 127.10.25.1:51911)
0806 19:40:27.444010 (+ 1870us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:27.444142 (+ 132us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:27.447271 (+ 3129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:27.448648 (+ 1377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:27.448660 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:27.448797 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:27.449792 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:27.449819 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:27.451621 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:27.451626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:27.453204 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:27.453226 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:27.454904 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:27.454911 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:27.455217 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:27.456320 (+ 1103us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:40:27.456324 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:27.456375 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:40:27.456494 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:27.457108 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:27.457121 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:27.457129 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 19:40:27.457839 (+ 710us) client_negotiation.cc:240] Negotiation successful
0806 19:40:27.458842 (+ 1003us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":489,"threads_started":1}
I20260806 19:40:27.462934 28871 heartbeater.cc:347] Connected to a master server at 127.10.25.62:35541
I20260806 19:40:27.463757 28871 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:27.465629 28871 heartbeater.cc:511] Master 127.10.25.62:35541 requested a full tablet report, sending...
I20260806 19:40:27.469764 28681 ts_manager.cc:194] Registered new tserver with Master: d20d37e070334c93af2417638a2ac308 (127.10.25.1:43793)
I20260806 19:40:27.471434 28681 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.25.1:51911
W20260806 19:40:27.891886 28875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:27.892278 28875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:27.892457 28875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:27.902478 28875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:27.902612 28875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:27.902912 28875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:40:27.914475 28875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:35541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:27.916710 28875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:27.919661 28875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:27.935814 28880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:27.935896 28881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:27.935925 28883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:27.938476 28875 server_base.cc:1034] running on GCE node
I20260806 19:40:27.939347 28875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:27.940521 28875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:27.941874 28875 hybrid_clock.cc:648] HybridClock initialized: now 1786045227941789 us; error 75 us; skew 500 ppm
I20260806 19:40:27.942337 28875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:40:27.946117 28875 webserver.cc:460] Webserver started at http://127.10.25.2:43217/ using document root <none> and password file <none>
I20260806 19:40:27.947069 28875 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:27.947198 28875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:27.947604 28875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:27.951419 28875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "82241789b0a347b4b1a0e07316a8ba5a"
format_stamp: "Formatted at 2026-08-06 19:40:27 on dist-test-slave-ss4n"
server_key: "6b62c63ab2264558fcf1e423dc0bac9d"
server_key_iv: "de4e3e520f959a4108c6955132669c53"
server_key_version: "encryptionkey@0"
I20260806 19:40:27.952420 28875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "82241789b0a347b4b1a0e07316a8ba5a"
format_stamp: "Formatted at 2026-08-06 19:40:27 on dist-test-slave-ss4n"
server_key: "6b62c63ab2264558fcf1e423dc0bac9d"
server_key_iv: "de4e3e520f959a4108c6955132669c53"
server_key_version: "encryptionkey@0"
I20260806 19:40:27.959316 28875 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 19:40:27.964133 28889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:27.966719 28875 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 19:40:27.967000 28875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "82241789b0a347b4b1a0e07316a8ba5a"
format_stamp: "Formatted at 2026-08-06 19:40:27 on dist-test-slave-ss4n"
server_key: "6b62c63ab2264558fcf1e423dc0bac9d"
server_key_iv: "de4e3e520f959a4108c6955132669c53"
server_key_version: "encryptionkey@0"
I20260806 19:40:27.967290 28875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:27.984130 28875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:27.985111 28875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:27.985558 28875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:27.986738 28875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:27.988402 28875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:27.988512 28875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:27.988675 28875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:27.988757 28875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:28.029851 28875 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:36033
I20260806 19:40:28.029965 29002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:36033 every 8 connection(s)
I20260806 19:40:28.031745 28875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:40:28.034811 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 28875
I20260806 19:40:28.035101 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:40:28.035673 10340 external_mini_cluster.cc:1428] Setting key 4148ec10980c6f72d6dbce09f62186b7
I20260806 19:40:28.039239 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:35541
--builtin_ntp_servers=127.10.25.20:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 19:40:28.049229 29005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.035449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:50325 (local address 127.10.25.62:35541)
0806 19:40:28.036204 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:28.036211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:28.037927 (+ 1716us) server_negotiation.cc:407] Connection header received
0806 19:40:28.040498 (+ 2571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:28.040506 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:28.040626 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:28.040804 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:28.042377 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.043248 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.045265 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.045843 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.046039 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.047886 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:28.047894 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:28.047898 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:28.047932 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:28.047990 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:28.048006 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:28.048134 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:28.048655 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 19:40:28.048801 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":394,"threads_started":1}
I20260806 19:40:28.051119 29006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.036234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:35541 (local address 127.10.25.2:50325)
0806 19:40:28.037681 (+ 1447us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:28.037746 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:28.040100 (+ 2354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:28.041139 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:28.041151 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:28.041270 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:28.042105 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:28.042129 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.043490 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.043495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:28.045038 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:28.045057 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.046173 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.046182 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:28.046488 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.047441 (+ 953us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:40:28.047444 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:28.047496 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:40:28.047600 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:28.048136 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:28.048153 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:28.048167 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 19:40:28.048565 (+ 398us) client_negotiation.cc:240] Negotiation successful
0806 19:40:28.049171 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":446,"threads_started":1}
I20260806 19:40:28.054459 29003 heartbeater.cc:347] Connected to a master server at 127.10.25.62:35541
I20260806 19:40:28.054903 29003 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:28.055925 29003 heartbeater.cc:511] Master 127.10.25.62:35541 requested a full tablet report, sending...
I20260806 19:40:28.058154 28681 ts_manager.cc:194] Registered new tserver with Master: 82241789b0a347b4b1a0e07316a8ba5a (127.10.25.2:36033)
I20260806 19:40:28.059054 28681 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.25.2:50325
W20260806 19:40:28.466549 29007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:28.466904 29007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:28.467072 29007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:28.476235 29007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:28.476423 29007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
I20260806 19:40:28.476377 28871 heartbeater.cc:503] Master 127.10.25.62:35541 was elected leader, sending a full tablet report...
W20260806 19:40:28.476619 29007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:40:28.487380 29007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:35541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:28.488794 29007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:28.490823 29007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:28.507647 29012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:28.508018 29013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:28.508080 29015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:28.510313 29007 server_base.cc:1034] running on GCE node
I20260806 19:40:28.511205 29007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:28.512445 29007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:28.513761 29007 hybrid_clock.cc:648] HybridClock initialized: now 1786045228513721 us; error 106 us; skew 500 ppm
I20260806 19:40:28.514174 29007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:40:28.517412 29007 webserver.cc:460] Webserver started at http://127.10.25.3:39937/ using document root <none> and password file <none>
I20260806 19:40:28.518347 29007 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:28.518514 29007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:28.518894 29007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:28.523646 29007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "32783b8ed77b465fbec1b550b3e4a23c"
format_stamp: "Formatted at 2026-08-06 19:40:28 on dist-test-slave-ss4n"
server_key: "a661a5526ba09c047757e26d672a5678"
server_key_iv: "81221de4cdff234bcb35a8d3b717c4b1"
server_key_version: "encryptionkey@0"
I20260806 19:40:28.524720 29007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "32783b8ed77b465fbec1b550b3e4a23c"
format_stamp: "Formatted at 2026-08-06 19:40:28 on dist-test-slave-ss4n"
server_key: "a661a5526ba09c047757e26d672a5678"
server_key_iv: "81221de4cdff234bcb35a8d3b717c4b1"
server_key_version: "encryptionkey@0"
I20260806 19:40:28.531320 29007 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.008s
I20260806 19:40:28.536810 29021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:28.539325 29007 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260806 19:40:28.539589 29007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "32783b8ed77b465fbec1b550b3e4a23c"
format_stamp: "Formatted at 2026-08-06 19:40:28 on dist-test-slave-ss4n"
server_key: "a661a5526ba09c047757e26d672a5678"
server_key_iv: "81221de4cdff234bcb35a8d3b717c4b1"
server_key_version: "encryptionkey@0"
I20260806 19:40:28.539865 29007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:28.570092 29007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:28.571384 29007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:28.571741 29007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:28.573011 29007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:28.575551 29007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:28.575781 29007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:28.575941 29007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:28.576045 29007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:28.621663 29007 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:46385
I20260806 19:40:28.621801 29134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:46385 every 8 connection(s)
I20260806 19:40:28.623268 29007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:40:28.632853 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 29007
I20260806 19:40:28.633111 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:40:28.633788 10340 external_mini_cluster.cc:1428] Setting key 8c4b8f78418ab62e5d7dc8474d007c52
I20260806 19:40:28.643066 29137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.627404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:45611 (local address 127.10.25.62:35541)
0806 19:40:28.628009 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:28.628014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:28.629222 (+ 1208us) server_negotiation.cc:407] Connection header received
0806 19:40:28.632048 (+ 2826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:28.632055 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:28.632244 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:28.632394 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:28.634565 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.635764 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.639157 (+ 3393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.639681 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.639815 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.641958 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:28.641964 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:28.641968 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:28.641991 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:28.642056 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:28.642071 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:28.642159 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:28.642616 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 19:40:28.642721 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":381,"threads_started":1}
I20260806 19:40:28.644305 29138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.627827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:35541 (local address 127.10.25.3:45611)
0806 19:40:28.629010 (+ 1183us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:28.629062 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:28.631535 (+ 2473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:28.633010 (+ 1475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:28.633029 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:28.633234 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:28.634175 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:28.634203 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.636122 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.636133 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:28.638914 (+ 2781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:28.638941 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.640109 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.640119 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:28.640416 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.641366 (+ 950us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:40:28.641369 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:28.641421 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:40:28.641618 (+ 197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:28.642248 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:28.642259 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:28.642268 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 19:40:28.642486 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 19:40:28.642911 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":355,"threads_started":1}
I20260806 19:40:28.647190 29135 heartbeater.cc:347] Connected to a master server at 127.10.25.62:35541
I20260806 19:40:28.647698 29135 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:28.648718 29135 heartbeater.cc:511] Master 127.10.25.62:35541 requested a full tablet report, sending...
I20260806 19:40:28.651084 28681 ts_manager.cc:194] Registered new tserver with Master: 32783b8ed77b465fbec1b550b3e4a23c (127.10.25.3:46385)
I20260806 19:40:28.652307 28681 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.25.3:45611
I20260806 19:40:28.653034 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:40:28.666317 29137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.658351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57302 (local address 127.10.25.62:35541)
0806 19:40:28.658647 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:28.658658 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:28.659268 (+ 610us) server_negotiation.cc:407] Connection header received
0806 19:40:28.659466 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:28.659470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:28.659558 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:28.659665 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:28.660241 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.661082 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.663289 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.663782 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.664012 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.665353 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:28.665360 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:28.665373 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:28.665396 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:28.665438 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:28.665453 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:28.665628 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:28.665793 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 19:40:28.665890 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 19:40:28.671434 28681 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:40:28.680905 28681 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:40:28.716817 29148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.698765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:46385 (local address 127.0.0.1:52416)
0806 19:40:28.701070 (+ 2305us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:28.701124 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:28.701343 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:28.703376 (+ 2033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:28.703388 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:28.703434 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:28.704023 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:28.704043 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.707844 (+ 3801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.707856 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:28.712558 (+ 4702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:28.712606 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.715312 (+ 2706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.715325 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:28.715661 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.715744 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 19:40:28.715860 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 19:40:28.716376 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1959,"thread_start_us":1734,"threads_started":1}
I20260806 19:40:28.717635 29149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.699227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52416 (local address 127.10.25.3:46385)
0806 19:40:28.702435 (+ 3208us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:28.702446 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:28.702491 (+ 45us) server_negotiation.cc:407] Connection header received
0806 19:40:28.702636 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:28.702644 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:28.702950 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:28.704304 (+ 1354us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:28.704728 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.707377 (+ 2649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.713145 (+ 5768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.715000 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.716101 (+ 1101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.716496 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:28.716589 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 19:40:28.716806 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2974,"thread_start_us":537,"threads_started":1}
I20260806 19:40:28.722537 29069 tablet_service.cc:1467] Processing CreateTablet for tablet 5e846817e58944ecb08645b87d8a131f (DEFAULT_TABLE table=test-table [id=7e9ff0a454b8415ba959722e91ad81c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:28.723973 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.701072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:36033 (local address 127.0.0.1:38562)
0806 19:40:28.702120 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:28.702142 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:28.702284 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:28.714916 (+ 12632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:28.714925 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:28.714967 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:28.715504 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:28.715528 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.717996 (+ 2468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.718005 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:28.720353 (+ 2348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:28.720367 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.722183 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.722191 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:28.722389 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.722428 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 19:40:28.722561 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 19:40:28.723249 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":905,"thread_start_us":724,"threads_started":1}
I20260806 19:40:28.724700 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.701134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38562 (local address 127.10.25.2:36033)
0806 19:40:28.714061 (+ 12927us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:28.714073 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:28.714112 (+ 39us) server_negotiation.cc:407] Connection header received
0806 19:40:28.714243 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:28.714253 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:28.714607 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:28.715785 (+ 1178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:28.716055 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.717800 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.720601 (+ 2801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.721908 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.722081 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.722374 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:28.722632 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 19:40:28.722846 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12722,"thread_start_us":12677,"threads_started":1}
I20260806 19:40:28.725296 29152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.702594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:43793 (local address 127.0.0.1:58928)
0806 19:40:28.714654 (+ 12060us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:28.714677 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:28.714862 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:28.715946 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:28.715951 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:28.715985 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:28.716492 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:28.716508 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.718796 (+ 2288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.718802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:28.721337 (+ 2535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:28.721355 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.724425 (+ 3070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.724434 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:28.724649 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.724677 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 19:40:28.724902 (+ 225us) client_negotiation.cc:240] Negotiation successful
0806 19:40:28.725048 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11900,"thread_start_us":11722,"threads_started":1}
I20260806 19:40:28.725692 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.702702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58928 (local address 127.10.25.1:43793)
0806 19:40:28.713768 (+ 11066us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:28.713783 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:28.715053 (+ 1270us) server_negotiation.cc:407] Connection header received
0806 19:40:28.715173 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:28.715183 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:28.715436 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:28.715684 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:28.716721 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.718585 (+ 1864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.721651 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.723547 (+ 1896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.723718 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.724011 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:28.725023 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0806 19:40:28.725232 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10734,"thread_start_us":10531,"threads_started":1}
I20260806 19:40:28.725749 29069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e846817e58944ecb08645b87d8a131f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:28.728879 28937 tablet_service.cc:1467] Processing CreateTablet for tablet 5e846817e58944ecb08645b87d8a131f (DEFAULT_TABLE table=test-table [id=7e9ff0a454b8415ba959722e91ad81c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:28.729933 28804 tablet_service.cc:1467] Processing CreateTablet for tablet 5e846817e58944ecb08645b87d8a131f (DEFAULT_TABLE table=test-table [id=7e9ff0a454b8415ba959722e91ad81c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:28.730858 28937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e846817e58944ecb08645b87d8a131f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:28.731570 28804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e846817e58944ecb08645b87d8a131f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:28.745812 29154 tablet_bootstrap.cc:492] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c: Bootstrap starting.
I20260806 19:40:28.747094 29155 tablet_bootstrap.cc:492] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308: Bootstrap starting.
I20260806 19:40:28.749948 29154 tablet_bootstrap.cc:654] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:28.751190 29156 tablet_bootstrap.cc:492] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a: Bootstrap starting.
I20260806 19:40:28.751220 29155 tablet_bootstrap.cc:654] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:28.751391 29154 log.cc:824] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:28.752717 29155 log.cc:824] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:28.757241 29156 tablet_bootstrap.cc:654] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:28.759684 29156 log.cc:824] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:28.764256 29155 tablet_bootstrap.cc:492] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308: No bootstrap required, opened a new log
I20260806 19:40:28.764256 29156 tablet_bootstrap.cc:492] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a: No bootstrap required, opened a new log
I20260806 19:40:28.764669 29155 ts_tablet_manager.cc:1397] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.010s
I20260806 19:40:28.764665 29156 ts_tablet_manager.cc:1397] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.006s
I20260806 19:40:28.764846 29154 tablet_bootstrap.cc:492] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c: No bootstrap required, opened a new log
I20260806 19:40:28.765422 29154 ts_tablet_manager.cc:1397] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.007s
I20260806 19:40:28.774583 29154 raft_consensus.cc:348] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32783b8ed77b465fbec1b550b3e4a23c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46385 } } peers { permanent_uuid: "d20d37e070334c93af2417638a2ac308" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43793 } } peers { permanent_uuid: "82241789b0a347b4b1a0e07316a8ba5a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 36033 } }
I20260806 19:40:28.775084 29154 raft_consensus.cc:374] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:28.775189 29154 raft_consensus.cc:729] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32783b8ed77b465fbec1b550b3e4a23c, State: Initialized, Role: FOLLOWER
I20260806 19:40:28.776090 29154 consensus_queue.cc:260] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32783b8ed77b465fbec1b550b3e4a23c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46385 } } peers { permanent_uuid: "d20d37e070334c93af2417638a2ac308" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43793 } } peers { permanent_uuid: "82241789b0a347b4b1a0e07316a8ba5a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 36033 } }
I20260806 19:40:28.776544 29155 raft_consensus.cc:348] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32783b8ed77b465fbec1b550b3e4a23c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46385 } } peers { permanent_uuid: "d20d37e070334c93af2417638a2ac308" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43793 } } peers { permanent_uuid: "82241789b0a347b4b1a0e07316a8ba5a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 36033 } }
I20260806 19:40:28.776540 29156 raft_consensus.cc:348] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32783b8ed77b465fbec1b550b3e4a23c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46385 } } peers { permanent_uuid: "d20d37e070334c93af2417638a2ac308" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43793 } } peers { permanent_uuid: "82241789b0a347b4b1a0e07316a8ba5a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 36033 } }
I20260806 19:40:28.777036 29156 raft_consensus.cc:374] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:28.777036 29155 raft_consensus.cc:374] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:28.777149 29155 raft_consensus.cc:729] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d20d37e070334c93af2417638a2ac308, State: Initialized, Role: FOLLOWER
I20260806 19:40:28.777159 29156 raft_consensus.cc:729] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82241789b0a347b4b1a0e07316a8ba5a, State: Initialized, Role: FOLLOWER
I20260806 19:40:28.777644 29135 heartbeater.cc:503] Master 127.10.25.62:35541 was elected leader, sending a full tablet report...
I20260806 19:40:28.777872 29155 consensus_queue.cc:260] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32783b8ed77b465fbec1b550b3e4a23c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46385 } } peers { permanent_uuid: "d20d37e070334c93af2417638a2ac308" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43793 } } peers { permanent_uuid: "82241789b0a347b4b1a0e07316a8ba5a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 36033 } }
I20260806 19:40:28.777873 29156 consensus_queue.cc:260] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32783b8ed77b465fbec1b550b3e4a23c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46385 } } peers { permanent_uuid: "d20d37e070334c93af2417638a2ac308" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43793 } } peers { permanent_uuid: "82241789b0a347b4b1a0e07316a8ba5a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 36033 } }
I20260806 19:40:28.778195 29154 ts_tablet_manager.cc:1428] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
I20260806 19:40:28.780421 29156 ts_tablet_manager.cc:1428] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a: Time spent starting tablet: real 0.016s user 0.007s sys 0.007s
I20260806 19:40:28.780596 29155 ts_tablet_manager.cc:1428] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260806 19:40:28.780967 29003 heartbeater.cc:503] Master 127.10.25.62:35541 was elected leader, sending a full tablet report...
W20260806 19:40:28.784531 29004 tablet.cc:2369] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:40:28.826223 29162 raft_consensus.cc:482] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:40:28.826740 29162 raft_consensus.cc:504] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32783b8ed77b465fbec1b550b3e4a23c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46385 } } peers { permanent_uuid: "d20d37e070334c93af2417638a2ac308" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43793 } } peers { permanent_uuid: "82241789b0a347b4b1a0e07316a8ba5a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 36033 } }
I20260806 19:40:28.828750 29162 leader_election.cc:288] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32783b8ed77b465fbec1b550b3e4a23c (127.10.25.3:46385), 82241789b0a347b4b1a0e07316a8ba5a (127.10.25.2:36033)
I20260806 19:40:28.837406 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.829435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:39753 (local address 127.10.25.2:36033)
0806 19:40:28.829719 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:28.829726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:28.831108 (+ 1382us) server_negotiation.cc:407] Connection header received
0806 19:40:28.831402 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:28.831406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:28.831483 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:28.831598 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:28.832527 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.833292 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.835578 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.836355 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.836512 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.836646 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:28.836946 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 19:40:28.837104 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260806 19:40:28.837802 29164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.829411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:36033 (local address 127.10.25.1:39753)
0806 19:40:28.830876 (+ 1465us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:28.830904 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:28.831239 (+ 335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:28.831664 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:28.831669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:28.831718 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:28.832348 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:28.832361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.833445 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.833451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:28.835329 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:28.835345 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.836582 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.836589 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:28.836770 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.836795 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 19:40:28.836913 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 19:40:28.837334 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1307,"mutex_wait_us":135,"thread_start_us":1000,"threads_started":1}
I20260806 19:40:28.838615 29149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.829471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:37089 (local address 127.10.25.3:46385)
0806 19:40:28.829750 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:28.829759 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:28.830469 (+ 710us) server_negotiation.cc:407] Connection header received
0806 19:40:28.830784 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:28.830790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:28.830891 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:28.831388 (+ 497us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:28.832359 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.833203 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.835816 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.836714 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.837083 (+ 369us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.837371 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:28.838159 (+ 788us) server_negotiation.cc:299] Negotiation successful
0806 19:40:28.838333 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 19:40:28.838680 29163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.829411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:46385 (local address 127.10.25.1:37089)
0806 19:40:28.830323 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:28.830345 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:28.830585 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:28.831423 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:28.831430 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:28.831473 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:28.832054 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:28.832071 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.833399 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.833406 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:28.835560 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:28.835577 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.837233 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.837371 (+ 138us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:28.837805 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.837844 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 19:40:28.838122 (+ 278us) client_negotiation.cc:240] Negotiation successful
0806 19:40:28.838279 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":134,"thread_start_us":539,"threads_started":1}
I20260806 19:40:28.839666 28957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e846817e58944ecb08645b87d8a131f" candidate_uuid: "d20d37e070334c93af2417638a2ac308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82241789b0a347b4b1a0e07316a8ba5a" is_pre_election: true
I20260806 19:40:28.839745 29089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e846817e58944ecb08645b87d8a131f" candidate_uuid: "d20d37e070334c93af2417638a2ac308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32783b8ed77b465fbec1b550b3e4a23c" is_pre_election: true
I20260806 19:40:28.840312 28957 raft_consensus.cc:2454] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d20d37e070334c93af2417638a2ac308 in term 0.
I20260806 19:40:28.840378 29089 raft_consensus.cc:2454] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d20d37e070334c93af2417638a2ac308 in term 0.
I20260806 19:40:28.841293 28757 leader_election.cc:302] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82241789b0a347b4b1a0e07316a8ba5a, d20d37e070334c93af2417638a2ac308; no voters:
I20260806 19:40:28.842001 29162 raft_consensus.cc:2781] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:40:28.842370 29162 raft_consensus.cc:482] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:40:28.842587 29162 raft_consensus.cc:3037] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:28.845408 29162 raft_consensus.cc:504] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32783b8ed77b465fbec1b550b3e4a23c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46385 } } peers { permanent_uuid: "d20d37e070334c93af2417638a2ac308" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43793 } } peers { permanent_uuid: "82241789b0a347b4b1a0e07316a8ba5a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 36033 } }
I20260806 19:40:28.846295 29162 leader_election.cc:288] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [CANDIDATE]: Term 1 election: Requested vote from peers 32783b8ed77b465fbec1b550b3e4a23c (127.10.25.3:46385), 82241789b0a347b4b1a0e07316a8ba5a (127.10.25.2:36033)
I20260806 19:40:28.847003 28957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e846817e58944ecb08645b87d8a131f" candidate_uuid: "d20d37e070334c93af2417638a2ac308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82241789b0a347b4b1a0e07316a8ba5a"
I20260806 19:40:28.846992 29089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e846817e58944ecb08645b87d8a131f" candidate_uuid: "d20d37e070334c93af2417638a2ac308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32783b8ed77b465fbec1b550b3e4a23c"
I20260806 19:40:28.847266 29089 raft_consensus.cc:3037] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:28.847266 28957 raft_consensus.cc:3037] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:28.851537 29089 raft_consensus.cc:2454] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d20d37e070334c93af2417638a2ac308 in term 1.
I20260806 19:40:28.851537 28957 raft_consensus.cc:2454] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d20d37e070334c93af2417638a2ac308 in term 1.
I20260806 19:40:28.852308 28757 leader_election.cc:302] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 32783b8ed77b465fbec1b550b3e4a23c, 82241789b0a347b4b1a0e07316a8ba5a, d20d37e070334c93af2417638a2ac308; no voters:
I20260806 19:40:28.852736 29162 raft_consensus.cc:2781] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:28.853327 29162 raft_consensus.cc:686] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [term 1 LEADER]: Becoming Leader. State: Replica: d20d37e070334c93af2417638a2ac308, State: Running, Role: LEADER
I20260806 19:40:28.853878 29162 consensus_queue.cc:237] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32783b8ed77b465fbec1b550b3e4a23c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46385 } } peers { permanent_uuid: "d20d37e070334c93af2417638a2ac308" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43793 } } peers { permanent_uuid: "82241789b0a347b4b1a0e07316a8ba5a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 36033 } }
I20260806 19:40:28.859822 28679 catalog_manager.cc:5495] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 reported cstate change: term changed from 0 to 1, leader changed from <none> to d20d37e070334c93af2417638a2ac308 (127.10.25.1). New cstate: current_term: 1 leader_uuid: "d20d37e070334c93af2417638a2ac308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32783b8ed77b465fbec1b550b3e4a23c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d20d37e070334c93af2417638a2ac308" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 43793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82241789b0a347b4b1a0e07316a8ba5a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 36033 } health_report { overall_health: UNKNOWN } } }
W20260806 19:40:28.875490 29136 tablet.cc:2369] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:40:28.889894 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.883177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58942 (local address 127.10.25.1:43793)
0806 19:40:28.883515 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:28.883521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:28.883581 (+ 60us) server_negotiation.cc:407] Connection header received
0806 19:40:28.883777 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:28.883782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:28.883881 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:28.884030 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:40:28.884435 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.885212 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.886903 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.887348 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.887460 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.888215 (+ 755us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:40:28.888771 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:40:28.888887 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:28.889364 (+ 477us) server_negotiation.cc:299] Negotiation successful
0806 19:40:28.889448 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260806 19:40:28.899021 28957 raft_consensus.cc:1261] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a [term 1 FOLLOWER]: Refusing update from remote peer d20d37e070334c93af2417638a2ac308: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:40:28.899021 29089 raft_consensus.cc:1261] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c [term 1 FOLLOWER]: Refusing update from remote peer d20d37e070334c93af2417638a2ac308: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:40:28.900206 29165 consensus_queue.cc:1035] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32783b8ed77b465fbec1b550b3e4a23c" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:40:28.900694 29162 consensus_queue.cc:1035] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82241789b0a347b4b1a0e07316a8ba5a" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 36033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:40:28.911428 29168 mvcc.cc:205] Tried to move back new op lower bound from 7315641257559015424 to 7315641257390067712. Current Snapshot: MvccSnapshot[applied={T|T < 7315641257559015424}]
I20260806 19:40:28.920867 29169 mvcc.cc:205] Tried to move back new op lower bound from 7315641257559015424 to 7315641257390067712. Current Snapshot: MvccSnapshot[applied={T|T < 7315641257559015424 or (T in {7315641257559015424})}]
I20260806 19:40:28.928592 28679 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:40:28.929025 28679 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:40:28.934294 28679 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a08d0da32e404e0fbe21330dc4fd10ff: Sending DeleteTablet for 3 replicas of tablet 5e846817e58944ecb08645b87d8a131f
I20260806 19:40:28.935714 28937 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e846817e58944ecb08645b87d8a131f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:28 UTC) from {username='slave'} at 127.0.0.1:38562
I20260806 19:40:28.935714 29069 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e846817e58944ecb08645b87d8a131f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:28 UTC) from {username='slave'} at 127.0.0.1:52416
I20260806 19:40:28.935853 28804 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e846817e58944ecb08645b87d8a131f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:28 UTC) from {username='slave'} at 127.0.0.1:58928
I20260806 19:40:28.941021 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 28741
I20260806 19:40:28.947970 29179 tablet_replica.cc:330] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308: stopping tablet replica
I20260806 19:40:28.948513 29179 raft_consensus.cc:2229] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:40:28.949290 29179 raft_consensus.cc:2258] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:28.951999 29179 ts_tablet_manager.cc:1905] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:28.952644 29178 tablet_replica.cc:330] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a: stopping tablet replica
I20260806 19:40:28.953154 29178 raft_consensus.cc:2229] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:40:28.953740 29178 raft_consensus.cc:2258] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:28.958820 29153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:28.944543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58946 (local address 127.10.25.1:43793)
0806 19:40:28.944805 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:28.944815 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:28.946459 (+ 1644us) server_negotiation.cc:407] Connection header received
0806 19:40:28.947579 (+ 1120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:28.947591 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:28.947726 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:28.947816 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:28.948670 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.949925 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.954220 (+ 4295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:28.954957 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:28.955066 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:28.956738 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:28.956750 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:28.956771 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:28.957719 (+ 948us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:28.957801 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:28.957828 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:28.957925 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:28.958226 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 19:40:28.958360 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 19:40:28.960258 28866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:28.962761 29178 ts_tablet_manager.cc:1905] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:28.963238 29177 tablet_replica.cc:330] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c: stopping tablet replica
I20260806 19:40:28.963899 29177 raft_consensus.cc:2229] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:40:28.964769 29177 raft_consensus.cc:2258] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:28.968483 29177 ts_tablet_manager.cc:1905] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:28.971413 29178 ts_tablet_manager.cc:1918] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:40:28.971627 29178 log.cc:1196] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/5e846817e58944ecb08645b87d8a131f
I20260806 19:40:28.972525 29178 ts_tablet_manager.cc:1939] T 5e846817e58944ecb08645b87d8a131f P 82241789b0a347b4b1a0e07316a8ba5a: Deleting consensus metadata
I20260806 19:40:28.974628 28666 catalog_manager.cc:4841] TS 82241789b0a347b4b1a0e07316a8ba5a (127.10.25.2:36033): tablet 5e846817e58944ecb08645b87d8a131f (table test-table [id=7e9ff0a454b8415ba959722e91ad81c8]) successfully deleted
I20260806 19:40:28.978433 29177 ts_tablet_manager.cc:1918] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:40:28.978696 29177 log.cc:1196] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/5e846817e58944ecb08645b87d8a131f
I20260806 19:40:28.979368 29177 ts_tablet_manager.cc:1939] T 5e846817e58944ecb08645b87d8a131f P 32783b8ed77b465fbec1b550b3e4a23c: Deleting consensus metadata
I20260806 19:40:28.982097 28668 catalog_manager.cc:4841] TS 32783b8ed77b465fbec1b550b3e4a23c (127.10.25.3:46385): tablet 5e846817e58944ecb08645b87d8a131f (table test-table [id=7e9ff0a454b8415ba959722e91ad81c8]) successfully deleted
I20260806 19:40:29.083995 29179 ts_tablet_manager.cc:1918] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:40:29.084426 29179 log.cc:1196] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/5e846817e58944ecb08645b87d8a131f
I20260806 19:40:29.085073 29179 ts_tablet_manager.cc:1939] T 5e846817e58944ecb08645b87d8a131f P d20d37e070334c93af2417638a2ac308: Deleting consensus metadata
I20260806 19:40:29.087034 28668 catalog_manager.cc:4841] TS d20d37e070334c93af2417638a2ac308 (127.10.25.1:43793): tablet 5e846817e58944ecb08645b87d8a131f (table test-table [id=7e9ff0a454b8415ba959722e91ad81c8]) successfully deleted
I20260806 19:40:29.087173 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 28741
I20260806 19:40:29.108063 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 28875
I20260806 19:40:29.116612 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:29.108904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38572 (local address 127.10.25.2:36033)
0806 19:40:29.109152 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:29.109162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:29.109240 (+ 78us) server_negotiation.cc:407] Connection header received
0806 19:40:29.109514 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:29.109521 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:29.109643 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:29.109774 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:29.110244 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:29.111127 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:29.112619 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:29.113084 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:29.113237 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:29.114639 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:29.114649 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:29.114667 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:29.115380 (+ 713us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:29.115437 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:29.115460 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:29.115614 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:29.115779 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 19:40:29.115907 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 19:40:29.117802 28998 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:29.220184 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 28875
I20260806 19:40:29.242362 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 29007
I20260806 19:40:29.250764 29149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:29.243282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52418 (local address 127.10.25.3:46385)
0806 19:40:29.243504 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:29.243508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:29.243529 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:40:29.243593 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:29.243597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:29.243703 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:29.243804 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:29.244313 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:29.245195 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:29.246990 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:29.247425 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:29.247537 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:29.249123 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:29.249134 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:29.249149 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:29.249912 (+ 763us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:29.249973 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:29.249997 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:29.250092 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:29.250370 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 19:40:29.250488 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 19:40:29.251492 29130 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:29.348114 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 29007
I20260806 19:40:29.367830 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 28650
I20260806 19:40:29.369438 28712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:29.486199 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 28650
2026-08-06T19:40:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3486 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:40:29 dist-test-slave-ss4n krb5kdc[29187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:40:29 dist-test-slave-ss4n krb5kdc[29187](info): set up 2 sockets
Aug 06 19:40:29 dist-test-slave-ss4n krb5kdc[29187](info): commencing operation
krb5kdc: starting...
W20260806 19:40:30.710945 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:40:30 dist-test-slave-ss4n krb5kdc[29187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:40:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:40:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:40:31.063181 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:41645
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:40843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:41645
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:31.443367 29200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:31.443711 29200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:31.443886 29200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:31.451774 29200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:40:31.451933 29200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:31.451985 29200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:31.452091 29200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:40:31.452171 29200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:40:31.465849 29200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:40843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:41645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:41645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:31.467340 29200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:31.469000 29200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:31.482442 29205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:31.482553 29208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:31.482530 29206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:31.483898 29200 server_base.cc:1034] running on GCE node
I20260806 19:40:31.484946 29200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:31.487382 29200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:31.488797 29200 hybrid_clock.cc:648] HybridClock initialized: now 1786045231488646 us; error 129 us; skew 500 ppm
I20260806 19:40:31.489353 29200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:31 dist-test-slave-ss4n krb5kdc[29187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045231, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:31.499526 29200 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:40:31.501768 29200 webserver.cc:460] Webserver started at http://127.10.25.62:44657/ using document root <none> and password file <none>
I20260806 19:40:31.502676 29200 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:31.502846 29200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:31.503254 29200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:31.506989 29200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "a0f6373f69cb41aab6737d863623ff02"
format_stamp: "Formatted at 2026-08-06 19:40:31 on dist-test-slave-ss4n"
server_key: "37f3997e5a1d14a5cb436f6f4041835e"
server_key_iv: "6f812f018bed1f1ad51492810bce2328"
server_key_version: "encryptionkey@0"
I20260806 19:40:31.507974 29200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "a0f6373f69cb41aab6737d863623ff02"
format_stamp: "Formatted at 2026-08-06 19:40:31 on dist-test-slave-ss4n"
server_key: "37f3997e5a1d14a5cb436f6f4041835e"
server_key_iv: "6f812f018bed1f1ad51492810bce2328"
server_key_version: "encryptionkey@0"
I20260806 19:40:31.515566 29200 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260806 19:40:31.521306 29215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:31.523162 29200 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 19:40:31.523432 29200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "a0f6373f69cb41aab6737d863623ff02"
format_stamp: "Formatted at 2026-08-06 19:40:31 on dist-test-slave-ss4n"
server_key: "37f3997e5a1d14a5cb436f6f4041835e"
server_key_iv: "6f812f018bed1f1ad51492810bce2328"
server_key_version: "encryptionkey@0"
I20260806 19:40:31.523706 29200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:31.547050 29200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:31.552791 29200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:31.553086 29200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:31.584115 29200 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:41645
I20260806 19:40:31.584246 29266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:41645 every 8 connection(s)
I20260806 19:40:31.585981 29200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:40:31.588802 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 29200
I20260806 19:40:31.589025 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:40:31.589855 10340 external_mini_cluster.cc:1428] Setting key 1dd9b35470373e8fe16945456a6ba974
I20260806 19:40:31.591169 29267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:31.606127 29267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02: Bootstrap starting.
Aug 06 19:40:31 dist-test-slave-ss4n krb5kdc[29187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:40:31.610467 29267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:31.611907 29267 log.cc:824] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:31.617847 29267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02: No bootstrap required, opened a new log
I20260806 19:40:31.619447 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:31.592172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43690 (local address 127.10.25.62:41645)
0806 19:40:31.593085 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:31.593099 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:31.593142 (+ 43us) server_negotiation.cc:407] Connection header received
0806 19:40:31.594967 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:31.594996 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:31.595484 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:31.595991 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:31.597800 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:31.599549 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:31.601266 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:31.602213 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:31.602489 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:31.607909 (+ 5420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:31.607919 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:31.607936 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:31.608007 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:31.612454 (+ 4447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:31.613755 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:31.613843 (+ 88us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:31.613923 (+ 80us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:31.614274 (+ 351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:31.615456 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:31.615461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:31.615463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:31.616362 (+ 899us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:31.616677 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:31.617848 (+ 1171us) server_negotiation.cc:299] Negotiation successful
0806 19:40:31.618329 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":435,"threads_started":1}
I20260806 19:40:31.627080 29267 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f6373f69cb41aab6737d863623ff02" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41645 } }
I20260806 19:40:31.627579 29267 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:31.627727 29267 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f6373f69cb41aab6737d863623ff02, State: Initialized, Role: FOLLOWER
I20260806 19:40:31.628504 29267 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f6373f69cb41aab6737d863623ff02" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41645 } }
I20260806 19:40:31.628752 29267 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:40:31.628907 29267 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:40:31.629117 29267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:31.631711 29267 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f6373f69cb41aab6737d863623ff02" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41645 } }
I20260806 19:40:31.632411 29267 leader_election.cc:302] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0f6373f69cb41aab6737d863623ff02; no voters:
I20260806 19:40:31.633272 29267 leader_election.cc:288] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:40:31.633409 29272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:31.634318 29272 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [term 1 LEADER]: Becoming Leader. State: Replica: a0f6373f69cb41aab6737d863623ff02, State: Running, Role: LEADER
I20260806 19:40:31.634976 29272 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f6373f69cb41aab6737d863623ff02" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41645 } }
I20260806 19:40:31.635629 29267 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:40:31.638307 29274 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0f6373f69cb41aab6737d863623ff02. Latest consensus state: current_term: 1 leader_uuid: "a0f6373f69cb41aab6737d863623ff02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f6373f69cb41aab6737d863623ff02" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41645 } } }
I20260806 19:40:31.638773 29274 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:31.639416 29273 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0f6373f69cb41aab6737d863623ff02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f6373f69cb41aab6737d863623ff02" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 41645 } } }
I20260806 19:40:31.639912 29273 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02 [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:31.642822 29280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:40:31.648682 29280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:40:31.661203 29280 catalog_manager.cc:1347] Generated new cluster ID: aefc43f759664f1aa7dc57aec561d853
I20260806 19:40:31.661365 29280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:40:31.687449 29280 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:40:31.688912 29280 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:40:31.696097 29280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02: Generated new TSK 0
I20260806 19:40:31.697219 29280 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:40:31.794893 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:41645
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:40843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:32.201874 29293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:32.202267 29293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:32.202440 29293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:32.210589 29293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:32.210788 29293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:32.210934 29293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:40:32.223017 29293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:40843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:41645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:32.224602 29293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:32.226301 29293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:32.243260 29299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:32.243314 29301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:32.243511 29298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:32.244230 29293 server_base.cc:1034] running on GCE node
I20260806 19:40:32.245756 29293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:32.247059 29293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:32.248353 29293 hybrid_clock.cc:648] HybridClock initialized: now 1786045232248260 us; error 92 us; skew 500 ppm
I20260806 19:40:32.248764 29293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:32 dist-test-slave-ss4n krb5kdc[29187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045232, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:32.252555 29293 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:40:32.254313 29293 webserver.cc:460] Webserver started at http://127.10.25.1:46225/ using document root <none> and password file <none>
I20260806 19:40:32.255056 29293 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:32.255168 29293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:32.255460 29293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:32.258771 29293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "7f39aec984404eaebec41f56031c112a"
format_stamp: "Formatted at 2026-08-06 19:40:32 on dist-test-slave-ss4n"
server_key: "cfca90f0b8d7c27fd7c2fce21950dd18"
server_key_iv: "ca593f7696ec7b8a7954619033f5240e"
server_key_version: "encryptionkey@0"
I20260806 19:40:32.259598 29293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "7f39aec984404eaebec41f56031c112a"
format_stamp: "Formatted at 2026-08-06 19:40:32 on dist-test-slave-ss4n"
server_key: "cfca90f0b8d7c27fd7c2fce21950dd18"
server_key_iv: "ca593f7696ec7b8a7954619033f5240e"
server_key_version: "encryptionkey@0"
I20260806 19:40:32.265727 29293 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260806 19:40:32.270015 29308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:32.271736 29293 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:40:32.271978 29293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "7f39aec984404eaebec41f56031c112a"
format_stamp: "Formatted at 2026-08-06 19:40:32 on dist-test-slave-ss4n"
server_key: "cfca90f0b8d7c27fd7c2fce21950dd18"
server_key_iv: "ca593f7696ec7b8a7954619033f5240e"
server_key_version: "encryptionkey@0"
I20260806 19:40:32.272235 29293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:32.302922 29293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:32.308548 29293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:32.308840 29293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:32.310177 29293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:32.311657 29293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:32.311738 29293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:32.311828 29293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:32.311873 29293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:32.354257 29293 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:41893
I20260806 19:40:32.354298 29420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:41893 every 8 connection(s)
I20260806 19:40:32.355705 29293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:40:32.364444 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 29293
I20260806 19:40:32.364714 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:40:32.365264 10340 external_mini_cluster.cc:1428] Setting key e5e0bada92fde855fde8d6c8337af732
Aug 06 19:40:32 dist-test-slave-ss4n krb5kdc[29187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045232, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:40:32.382881 29423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:32.358895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:47045 (local address 127.10.25.62:41645)
0806 19:40:32.359774 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:32.359780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:32.360668 (+ 888us) server_negotiation.cc:407] Connection header received
0806 19:40:32.362936 (+ 2268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:32.362943 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:32.363047 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:32.363190 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:32.365474 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:32.366478 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:32.368811 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:32.369251 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:32.369402 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:32.373867 (+ 4465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:32.373872 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:32.373876 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:32.373915 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:32.378185 (+ 4270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:32.379567 (+ 1382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:32.379572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:32.379575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:32.379750 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:32.380394 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:32.380398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:32.380400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:32.380789 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:32.381017 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:32.381925 (+ 908us) server_negotiation.cc:299] Negotiation successful
0806 19:40:32.382298 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":618,"threads_started":1}
I20260806 19:40:32.383610 29424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:32.359659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:41645 (local address 127.10.25.1:47045)
0806 19:40:32.360450 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:32.360506 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:32.362576 (+ 2070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:32.363486 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:32.363495 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:32.364159 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:32.365217 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:32.365243 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:32.366666 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:32.366670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:32.368619 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:32.368637 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:32.369566 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:32.369576 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:32.369836 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:32.370683 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:40:32.370709 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:32.373586 (+ 2877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:32.378428 (+ 4842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:32.378443 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:32.378468 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:32.379387 (+ 919us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:32.379905 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:32.379914 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:32.379920 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:32.380243 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:32.380982 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:32.380990 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:32.381396 (+ 406us) client_negotiation.cc:769] Sending connection context
0806 19:40:32.381776 (+ 380us) client_negotiation.cc:240] Negotiation successful
0806 19:40:32.382286 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":317,"threads_started":1}
I20260806 19:40:32.385731 29421 heartbeater.cc:347] Connected to a master server at 127.10.25.62:41645
I20260806 19:40:32.386113 29421 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:32.387015 29421 heartbeater.cc:511] Master 127.10.25.62:41645 requested a full tablet report, sending...
I20260806 19:40:32.389835 29232 ts_manager.cc:194] Registered new tserver with Master: 7f39aec984404eaebec41f56031c112a (127.10.25.1:41893)
I20260806 19:40:32.391606 29232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:47045
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:40:32.446290 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:41645
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:40843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:32.833098 29427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:32.833503 29427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:32.833734 29427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:32.842917 29427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:32.843127 29427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:32.843343 29427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:40:32.855640 29427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:40843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:41645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:32.857306 29427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:32.859031 29427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:32.874842 29433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:32.874841 29434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:32.874890 29436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:32.876516 29427 server_base.cc:1034] running on GCE node
I20260806 19:40:32.877811 29427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:32.879460 29427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:32.880782 29427 hybrid_clock.cc:648] HybridClock initialized: now 1786045232880737 us; error 41 us; skew 500 ppm
I20260806 19:40:32.881227 29427 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:32 dist-test-slave-ss4n krb5kdc[29187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045232, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:32.885543 29427 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:40:32.887400 29427 webserver.cc:460] Webserver started at http://127.10.25.2:33953/ using document root <none> and password file <none>
I20260806 19:40:32.888202 29427 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:32.888326 29427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:32.888665 29427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:32.892277 29427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "4a01692d62ce42a9818fda86492823e9"
format_stamp: "Formatted at 2026-08-06 19:40:32 on dist-test-slave-ss4n"
server_key: "20ca228e240818daa660fe2efb472815"
server_key_iv: "a5b986e5b6442f41c93fec7cae1d3c8c"
server_key_version: "encryptionkey@0"
I20260806 19:40:32.893199 29427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "4a01692d62ce42a9818fda86492823e9"
format_stamp: "Formatted at 2026-08-06 19:40:32 on dist-test-slave-ss4n"
server_key: "20ca228e240818daa660fe2efb472815"
server_key_iv: "a5b986e5b6442f41c93fec7cae1d3c8c"
server_key_version: "encryptionkey@0"
I20260806 19:40:32.900077 29427 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 19:40:32.905046 29443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:32.907310 29427 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 19:40:32.907512 29427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "4a01692d62ce42a9818fda86492823e9"
format_stamp: "Formatted at 2026-08-06 19:40:32 on dist-test-slave-ss4n"
server_key: "20ca228e240818daa660fe2efb472815"
server_key_iv: "a5b986e5b6442f41c93fec7cae1d3c8c"
server_key_version: "encryptionkey@0"
I20260806 19:40:32.907877 29427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:32.928077 29427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:32.934917 29427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:32.935238 29427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:32.936647 29427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:32.938598 29427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:32.938692 29427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:32.938843 29427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:32.938906 29427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:32.981426 29427 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:45653
I20260806 19:40:32.981518 29556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:45653 every 8 connection(s)
I20260806 19:40:32.982931 29427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:40:32.992309 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 29427
I20260806 19:40:32.992503 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:40:32.993070 10340 external_mini_cluster.cc:1428] Setting key 0ae008a40e2232f08c4ad404d16d023f
Aug 06 19:40:32 dist-test-slave-ss4n krb5kdc[29187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045232, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:40:33.008828 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:32.985961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:59281 (local address 127.10.25.62:41645)
0806 19:40:32.986527 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:32.986535 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:32.987580 (+ 1045us) server_negotiation.cc:407] Connection header received
0806 19:40:32.989632 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:32.989637 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:32.989764 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:32.989884 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:32.991856 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:32.993274 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:32.995681 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:32.996293 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:32.996417 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:33.001624 (+ 5207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:33.001632 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:33.001635 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:33.001680 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:33.004848 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:33.005989 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:33.005999 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:33.006007 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:33.006172 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:33.006719 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:33.006724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:33.006726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:33.007059 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:33.007266 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:33.008088 (+ 822us) server_negotiation.cc:299] Negotiation successful
0806 19:40:33.008365 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":264,"threads_started":1}
I20260806 19:40:33.010197 29560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:32.986539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:41645 (local address 127.10.25.2:59281)
0806 19:40:32.987393 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:32.987439 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:32.989199 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:32.990160 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:32.990170 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:32.990839 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:32.991473 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:32.991495 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:32.993647 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:32.993653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:32.995428 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:32.995445 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:32.996594 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:32.996603 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:32.996948 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:32.997932 (+ 984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:40:32.997977 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:33.001228 (+ 3251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:33.005099 (+ 3871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:33.005111 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:33.005140 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:33.005825 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:33.006312 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:33.006318 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:33.006322 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:33.006564 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:33.007216 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:33.007229 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:33.007661 (+ 432us) client_negotiation.cc:769] Sending connection context
0806 19:40:33.007987 (+ 326us) client_negotiation.cc:240] Negotiation successful
0806 19:40:33.008633 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":356,"threads_started":1}
I20260806 19:40:33.012214 29557 heartbeater.cc:347] Connected to a master server at 127.10.25.62:41645
I20260806 19:40:33.012610 29557 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:33.013727 29557 heartbeater.cc:511] Master 127.10.25.62:41645 requested a full tablet report, sending...
I20260806 19:40:33.015692 29232 ts_manager.cc:194] Registered new tserver with Master: 4a01692d62ce42a9818fda86492823e9 (127.10.25.2:45653)
I20260806 19:40:33.017128 29232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:59281
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:40:33.073024 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:41645
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:40843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:40:33.396765 29421 heartbeater.cc:503] Master 127.10.25.62:41645 was elected leader, sending a full tablet report...
W20260806 19:40:33.467437 29563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:33.468186 29563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:33.468514 29563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:33.477371 29563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:33.477644 29563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:33.477876 29563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:40:33.490643 29563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:40843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:41645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:33.492200 29563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:33.493746 29563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:33.509279 29569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:33.509299 29568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:33.509614 29571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:33.510807 29563 server_base.cc:1034] running on GCE node
I20260806 19:40:33.511727 29563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:33.512949 29563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:33.514209 29563 hybrid_clock.cc:648] HybridClock initialized: now 1786045233514137 us; error 74 us; skew 500 ppm
I20260806 19:40:33.514601 29563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:33 dist-test-slave-ss4n krb5kdc[29187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045233, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:33.518301 29563 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:40:33.520133 29563 webserver.cc:460] Webserver started at http://127.10.25.3:41043/ using document root <none> and password file <none>
I20260806 19:40:33.520880 29563 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:33.521003 29563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:33.521317 29563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:33.524612 29563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "79f691146e6f45018d837e236c65f2e3"
format_stamp: "Formatted at 2026-08-06 19:40:33 on dist-test-slave-ss4n"
server_key: "520650c332e2d813e7da0299d42d1c0a"
server_key_iv: "6fcbd6880c926d4ecc015ac6ba853425"
server_key_version: "encryptionkey@0"
I20260806 19:40:33.525439 29563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "79f691146e6f45018d837e236c65f2e3"
format_stamp: "Formatted at 2026-08-06 19:40:33 on dist-test-slave-ss4n"
server_key: "520650c332e2d813e7da0299d42d1c0a"
server_key_iv: "6fcbd6880c926d4ecc015ac6ba853425"
server_key_version: "encryptionkey@0"
I20260806 19:40:33.531877 29563 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 19:40:33.536381 29578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:33.538645 29563 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:40:33.538902 29563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "79f691146e6f45018d837e236c65f2e3"
format_stamp: "Formatted at 2026-08-06 19:40:33 on dist-test-slave-ss4n"
server_key: "520650c332e2d813e7da0299d42d1c0a"
server_key_iv: "6fcbd6880c926d4ecc015ac6ba853425"
server_key_version: "encryptionkey@0"
I20260806 19:40:33.539170 29563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:33.555972 29563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:33.562311 29563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:33.562676 29563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:33.563898 29563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:33.565462 29563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:33.565635 29563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 19:40:33.565793 29563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:33.565872 29563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:33.605400 29563 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:41569
I20260806 19:40:33.605594 29693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:41569 every 8 connection(s)
I20260806 19:40:33.606894 29563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:40:33.608531 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 29563
I20260806 19:40:33.608768 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:40:33.609269 10340 external_mini_cluster.cc:1428] Setting key 782c7ae918c8f239cdf028b3fe073620
Aug 06 19:40:33 dist-test-slave-ss4n krb5kdc[29187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045233, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:40:33.634688 29696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:33.610671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:51515 (local address 127.10.25.62:41645)
0806 19:40:33.611439 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:33.611445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:33.613572 (+ 2127us) server_negotiation.cc:407] Connection header received
0806 19:40:33.614842 (+ 1270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:33.614848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:33.614953 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:33.615104 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:33.616975 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.617995 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.620071 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.620529 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.620679 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:33.625601 (+ 4922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:33.625612 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:33.625616 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:33.625690 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:33.630074 (+ 4384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:33.631712 (+ 1638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:33.631719 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:33.631721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:33.631960 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:33.632559 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:33.632565 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:33.632569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:33.632906 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:33.633104 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:33.633960 (+ 856us) server_negotiation.cc:299] Negotiation successful
0806 19:40:33.634266 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":453,"threads_started":1}
I20260806 19:40:33.635763 29697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:33.611195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:41645 (local address 127.10.25.3:51515)
0806 19:40:33.612299 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:33.612356 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:33.614506 (+ 2150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:33.615412 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:33.615423 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:33.616068 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:33.616750 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:33.616772 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.618201 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.618206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:33.619862 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:33.619880 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.620805 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.620815 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:33.621075 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:33.621976 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:40:33.622002 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:33.625029 (+ 3027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:33.630347 (+ 5318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:33.630361 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:33.630385 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:33.631431 (+ 1046us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:33.632131 (+ 700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:33.632136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:33.632140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:33.632417 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:33.633051 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:33.633063 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:33.633551 (+ 488us) client_negotiation.cc:769] Sending connection context
0806 19:40:33.633867 (+ 316us) client_negotiation.cc:240] Negotiation successful
0806 19:40:33.634493 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"thread_start_us":454,"threads_started":1}
I20260806 19:40:33.638348 29694 heartbeater.cc:347] Connected to a master server at 127.10.25.62:41645
I20260806 19:40:33.638871 29694 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:33.640435 29694 heartbeater.cc:511] Master 127.10.25.62:41645 requested a full tablet report, sending...
I20260806 19:40:33.642849 29232 ts_manager.cc:194] Registered new tserver with Master: 79f691146e6f45018d837e236c65f2e3 (127.10.25.3:41569)
I20260806 19:40:33.644032 29232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:51515
I20260806 19:40:33.654767 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:40:33.678411 29696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:33.660220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43178 (local address 127.10.25.62:41645)
0806 19:40:33.660535 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:33.660553 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:33.660858 (+ 305us) server_negotiation.cc:407] Connection header received
0806 19:40:33.661023 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:33.661028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:33.661121 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:33.661226 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:33.663133 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.663905 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.665651 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.666323 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.666474 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:33.670336 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:33.670343 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:33.670347 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:33.670397 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:33.673910 (+ 3513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:33.675540 (+ 1630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:33.675546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:33.675548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:33.675758 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:33.676568 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:33.676572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:33.676575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:33.676948 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:33.677326 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:33.677826 (+ 500us) server_negotiation.cc:299] Negotiation successful
0806 19:40:33.678057 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260806 19:40:33.684402 29232 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:40:33.692694 29232 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:40:33.735693 29711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:33.709895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:41569 (local address 127.0.0.1:35904)
0806 19:40:33.724890 (+ 14995us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:33.724926 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:33.725151 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:33.727850 (+ 2699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:33.727860 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:33.727896 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:33.728453 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:33.728470 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.730504 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.730510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:33.732872 (+ 2362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:33.732889 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.734552 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.734559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:33.734760 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:33.734829 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 19:40:33.735023 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 19:40:33.735244 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14754,"mutex_wait_us":3,"thread_start_us":17092,"threads_started":1}
I20260806 19:40:33.736640 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:33.710753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35904 (local address 127.10.25.3:41569)
0806 19:40:33.726804 (+ 16051us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:33.726817 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:33.726870 (+ 53us) server_negotiation.cc:407] Connection header received
0806 19:40:33.727023 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:33.727038 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:33.727513 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:33.727841 (+ 328us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:33.728743 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.730315 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.733088 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.734291 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.734479 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:33.734750 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:33.735074 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 19:40:33.735263 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15796,"thread_start_us":15554,"threads_started":1}
I20260806 19:40:33.737689 29707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:33.709555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:45653 (local address 127.0.0.1:58482)
0806 19:40:33.710826 (+ 1271us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:33.710874 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:33.711070 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:33.728254 (+ 17184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:33.728262 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:33.728302 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:33.728933 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:33.728946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.731243 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.731252 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:33.734360 (+ 3108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:33.734379 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.736678 (+ 2299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.736688 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:33.736898 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:33.736938 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 19:40:33.737018 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 19:40:33.737187 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1019,"mutex_wait_us":125,"thread_start_us":866,"threads_started":1}
I20260806 19:40:33.738718 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:33.709869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58482 (local address 127.10.25.2:45653)
0806 19:40:33.727621 (+ 17752us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:33.727629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:33.727655 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:40:33.727773 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:33.727782 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:33.728016 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:33.728256 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:33.729147 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.731031 (+ 1884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.734635 (+ 3604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.736382 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.737866 (+ 1484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:33.738101 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:33.738195 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 19:40:33.738378 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17581,"thread_start_us":17452,"threads_started":1}
I20260806 19:40:33.739214 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:33.711309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:41893 (local address 127.0.0.1:56650)
0806 19:40:33.727296 (+ 15987us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:33.727320 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:33.727633 (+ 313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:33.728527 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:33.728533 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:33.728560 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:33.729152 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:33.729169 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.731400 (+ 2231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.731407 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:33.734371 (+ 2964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:33.734380 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.738109 (+ 3729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.738115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:33.738259 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:33.738285 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 19:40:33.738419 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 19:40:33.738576 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15836,"mutex_wait_us":108,"thread_start_us":14540,"threads_started":1}
I20260806 19:40:33.739714 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:33.711764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56650 (local address 127.10.25.1:41893)
0806 19:40:33.725939 (+ 14175us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:33.725950 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:33.727570 (+ 1620us) server_negotiation.cc:407] Connection header received
0806 19:40:33.727845 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:33.727852 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:33.728060 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:33.728237 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:33.729367 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.731213 (+ 1846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.734622 (+ 3409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.737285 (+ 2663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.737435 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:33.737753 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:33.738474 (+ 721us) server_negotiation.cc:299] Negotiation successful
0806 19:40:33.738677 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13963,"thread_start_us":13771,"threads_started":1}
I20260806 19:40:33.741173 29626 tablet_service.cc:1467] Processing CreateTablet for tablet 0eebf031c08d48a99606c6f87cd1a240 (DEFAULT_TABLE table=test-table [id=019babad61b74e2187e4c90f5e6ab570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:33.743026 29492 tablet_service.cc:1467] Processing CreateTablet for tablet 0eebf031c08d48a99606c6f87cd1a240 (DEFAULT_TABLE table=test-table [id=019babad61b74e2187e4c90f5e6ab570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:33.743853 29626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eebf031c08d48a99606c6f87cd1a240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:33.743798 29356 tablet_service.cc:1467] Processing CreateTablet for tablet 0eebf031c08d48a99606c6f87cd1a240 (DEFAULT_TABLE table=test-table [id=019babad61b74e2187e4c90f5e6ab570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:33.744627 29492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eebf031c08d48a99606c6f87cd1a240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:33.745379 29356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eebf031c08d48a99606c6f87cd1a240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:33.760700 29713 tablet_bootstrap.cc:492] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a: Bootstrap starting.
I20260806 19:40:33.761824 29714 tablet_bootstrap.cc:492] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9: Bootstrap starting.
I20260806 19:40:33.763864 29715 tablet_bootstrap.cc:492] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3: Bootstrap starting.
I20260806 19:40:33.766264 29713 tablet_bootstrap.cc:654] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:33.766268 29714 tablet_bootstrap.cc:654] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:33.767066 29715 tablet_bootstrap.cc:654] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:33.768280 29715 log.cc:824] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:33.768746 29713 log.cc:824] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:33.768746 29714 log.cc:824] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:33.772068 29715 tablet_bootstrap.cc:492] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3: No bootstrap required, opened a new log
I20260806 19:40:33.772432 29715 ts_tablet_manager.cc:1397] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 19:40:33.772751 29714 tablet_bootstrap.cc:492] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9: No bootstrap required, opened a new log
I20260806 19:40:33.773070 29714 ts_tablet_manager.cc:1397] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260806 19:40:33.774262 29713 tablet_bootstrap.cc:492] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a: No bootstrap required, opened a new log
I20260806 19:40:33.774606 29713 ts_tablet_manager.cc:1397] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.003s
I20260806 19:40:33.780642 29715 raft_consensus.cc:348] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a01692d62ce42a9818fda86492823e9" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45653 } } peers { permanent_uuid: "79f691146e6f45018d837e236c65f2e3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41569 } } peers { permanent_uuid: "7f39aec984404eaebec41f56031c112a" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 41893 } }
I20260806 19:40:33.781123 29715 raft_consensus.cc:374] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:33.781230 29715 raft_consensus.cc:729] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79f691146e6f45018d837e236c65f2e3, State: Initialized, Role: FOLLOWER
I20260806 19:40:33.782007 29715 consensus_queue.cc:260] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a01692d62ce42a9818fda86492823e9" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45653 } } peers { permanent_uuid: "79f691146e6f45018d837e236c65f2e3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41569 } } peers { permanent_uuid: "7f39aec984404eaebec41f56031c112a" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 41893 } }
I20260806 19:40:33.783255 29694 heartbeater.cc:503] Master 127.10.25.62:41645 was elected leader, sending a full tablet report...
I20260806 19:40:33.784039 29715 ts_tablet_manager.cc:1428] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260806 19:40:33.785755 29714 raft_consensus.cc:348] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a01692d62ce42a9818fda86492823e9" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45653 } } peers { permanent_uuid: "79f691146e6f45018d837e236c65f2e3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41569 } } peers { permanent_uuid: "7f39aec984404eaebec41f56031c112a" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 41893 } }
I20260806 19:40:33.786319 29714 raft_consensus.cc:374] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:33.786427 29714 raft_consensus.cc:729] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a01692d62ce42a9818fda86492823e9, State: Initialized, Role: FOLLOWER
I20260806 19:40:33.787182 29714 consensus_queue.cc:260] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a01692d62ce42a9818fda86492823e9" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45653 } } peers { permanent_uuid: "79f691146e6f45018d837e236c65f2e3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41569 } } peers { permanent_uuid: "7f39aec984404eaebec41f56031c112a" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 41893 } }
I20260806 19:40:33.787276 29713 raft_consensus.cc:348] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a01692d62ce42a9818fda86492823e9" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45653 } } peers { permanent_uuid: "79f691146e6f45018d837e236c65f2e3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41569 } } peers { permanent_uuid: "7f39aec984404eaebec41f56031c112a" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 41893 } }
I20260806 19:40:33.787750 29713 raft_consensus.cc:374] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:33.787900 29713 raft_consensus.cc:729] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f39aec984404eaebec41f56031c112a, State: Initialized, Role: FOLLOWER
I20260806 19:40:33.788618 29557 heartbeater.cc:503] Master 127.10.25.62:41645 was elected leader, sending a full tablet report...
I20260806 19:40:33.788975 29713 consensus_queue.cc:260] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a01692d62ce42a9818fda86492823e9" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45653 } } peers { permanent_uuid: "79f691146e6f45018d837e236c65f2e3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41569 } } peers { permanent_uuid: "7f39aec984404eaebec41f56031c112a" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 41893 } }
I20260806 19:40:33.789757 29714 ts_tablet_manager.cc:1428] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9: Time spent starting tablet: real 0.017s user 0.015s sys 0.004s
I20260806 19:40:33.792122 29713 ts_tablet_manager.cc:1428] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a: Time spent starting tablet: real 0.017s user 0.011s sys 0.007s
I20260806 19:40:33.832068 29720 raft_consensus.cc:482] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:40:33.832420 29720 raft_consensus.cc:504] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a01692d62ce42a9818fda86492823e9" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45653 } } peers { permanent_uuid: "79f691146e6f45018d837e236c65f2e3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41569 } } peers { permanent_uuid: "7f39aec984404eaebec41f56031c112a" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 41893 } }
I20260806 19:40:33.834599 29720 leader_election.cc:288] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79f691146e6f45018d837e236c65f2e3 (127.10.25.3:41569), 7f39aec984404eaebec41f56031c112a (127.10.25.1:41893)
I20260806 19:40:33.846287 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:33.835330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:44685 (local address 127.10.25.3:41569)
0806 19:40:33.835540 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:33.835547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:33.836982 (+ 1435us) server_negotiation.cc:407] Connection header received
0806 19:40:33.837412 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:33.837418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:33.837561 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:33.837660 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:33.838808 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.840000 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.843570 (+ 3570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.844907 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.845062 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:33.845180 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:33.845852 (+ 672us) server_negotiation.cc:299] Negotiation successful
0806 19:40:33.846009 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 19:40:33.846290 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:33.835238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:43449 (local address 127.10.25.1:41893)
0806 19:40:33.835540 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:33.835547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:33.836982 (+ 1435us) server_negotiation.cc:407] Connection header received
0806 19:40:33.837412 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:33.837418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:33.837547 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:33.837660 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:33.838808 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.840000 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.843542 (+ 3542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.844891 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.845070 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:33.845180 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:33.845855 (+ 675us) server_negotiation.cc:299] Negotiation successful
0806 19:40:33.846012 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260806 19:40:33.846524 29722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:33.835153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:41569 (local address 127.10.25.2:44685)
0806 19:40:33.836769 (+ 1616us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:33.836799 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:33.837072 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:33.837718 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:33.837724 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:33.837767 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:33.838588 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:33.838597 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.840310 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.840317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:33.843262 (+ 2945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:33.843286 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.845247 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.845252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:33.845506 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:33.845657 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 19:40:33.845769 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 19:40:33.845998 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1409,"mutex_wait_us":382,"spinlock_wait_cycles":69248,"thread_start_us":742,"threads_started":1}
I20260806 19:40:33.846660 29723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:33.835172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:41893 (local address 127.10.25.2:43449)
0806 19:40:33.836802 (+ 1630us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:33.836819 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:33.837022 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:33.837718 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:33.837724 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:33.837775 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:33.838580 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:33.838596 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.840311 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.840317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:33.843262 (+ 2945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:33.843286 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.845227 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.845238 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:33.845509 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:33.845657 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 19:40:33.845769 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 19:40:33.845995 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1509,"mutex_wait_us":563,"spinlock_wait_cycles":6400,"thread_start_us":698,"threads_started":1}
I20260806 19:40:33.847813 29646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eebf031c08d48a99606c6f87cd1a240" candidate_uuid: "4a01692d62ce42a9818fda86492823e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79f691146e6f45018d837e236c65f2e3" is_pre_election: true
I20260806 19:40:33.847867 29376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eebf031c08d48a99606c6f87cd1a240" candidate_uuid: "4a01692d62ce42a9818fda86492823e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f39aec984404eaebec41f56031c112a" is_pre_election: true
I20260806 19:40:33.848491 29376 raft_consensus.cc:2454] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a01692d62ce42a9818fda86492823e9 in term 0.
I20260806 19:40:33.848489 29646 raft_consensus.cc:2454] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a01692d62ce42a9818fda86492823e9 in term 0.
I20260806 19:40:33.849305 29444 leader_election.cc:302] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4a01692d62ce42a9818fda86492823e9, 79f691146e6f45018d837e236c65f2e3, 7f39aec984404eaebec41f56031c112a; no voters:
I20260806 19:40:33.849804 29720 raft_consensus.cc:2781] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:40:33.849962 29720 raft_consensus.cc:482] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:40:33.850075 29720 raft_consensus.cc:3037] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:33.852535 29720 raft_consensus.cc:504] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a01692d62ce42a9818fda86492823e9" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45653 } } peers { permanent_uuid: "79f691146e6f45018d837e236c65f2e3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41569 } } peers { permanent_uuid: "7f39aec984404eaebec41f56031c112a" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 41893 } }
I20260806 19:40:33.853271 29720 leader_election.cc:288] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [CANDIDATE]: Term 1 election: Requested vote from peers 79f691146e6f45018d837e236c65f2e3 (127.10.25.3:41569), 7f39aec984404eaebec41f56031c112a (127.10.25.1:41893)
I20260806 19:40:33.853905 29646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eebf031c08d48a99606c6f87cd1a240" candidate_uuid: "4a01692d62ce42a9818fda86492823e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79f691146e6f45018d837e236c65f2e3"
I20260806 19:40:33.854112 29646 raft_consensus.cc:3037] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:33.854017 29376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eebf031c08d48a99606c6f87cd1a240" candidate_uuid: "4a01692d62ce42a9818fda86492823e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f39aec984404eaebec41f56031c112a"
I20260806 19:40:33.854635 29376 raft_consensus.cc:3037] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:33.856781 29646 raft_consensus.cc:2454] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a01692d62ce42a9818fda86492823e9 in term 1.
I20260806 19:40:33.857163 29376 raft_consensus.cc:2454] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a01692d62ce42a9818fda86492823e9 in term 1.
I20260806 19:40:33.857349 29446 leader_election.cc:302] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a01692d62ce42a9818fda86492823e9, 79f691146e6f45018d837e236c65f2e3; no voters:
I20260806 19:40:33.857872 29720 raft_consensus.cc:2781] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:33.858711 29720 raft_consensus.cc:686] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [term 1 LEADER]: Becoming Leader. State: Replica: 4a01692d62ce42a9818fda86492823e9, State: Running, Role: LEADER
W20260806 19:40:33.859200 29695 tablet.cc:2369] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:40:33.859738 29720 consensus_queue.cc:237] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a01692d62ce42a9818fda86492823e9" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45653 } } peers { permanent_uuid: "79f691146e6f45018d837e236c65f2e3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41569 } } peers { permanent_uuid: "7f39aec984404eaebec41f56031c112a" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 41893 } }
W20260806 19:40:33.860206 29422 tablet.cc:2369] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:40:33.866549 29231 catalog_manager.cc:5495] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a01692d62ce42a9818fda86492823e9 (127.10.25.2). New cstate: current_term: 1 leader_uuid: "4a01692d62ce42a9818fda86492823e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a01692d62ce42a9818fda86492823e9" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 45653 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79f691146e6f45018d837e236c65f2e3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f39aec984404eaebec41f56031c112a" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 41893 } health_report { overall_health: UNKNOWN } } }
I20260806 19:40:33.895527 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:33.889253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58484 (local address 127.10.25.2:45653)
0806 19:40:33.889550 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:33.889558 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:33.889584 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:40:33.889734 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:33.889738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:33.889841 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:33.890040 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:40:33.890494 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.891249 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.892731 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.893159 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.893301 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:33.894037 (+ 736us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:40:33.894702 (+ 665us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:40:33.894976 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:33.895177 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 19:40:33.895236 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 19:40:33.906167 29376 raft_consensus.cc:1261] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a [term 1 FOLLOWER]: Refusing update from remote peer 4a01692d62ce42a9818fda86492823e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:40:33.906172 29646 raft_consensus.cc:1261] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3 [term 1 FOLLOWER]: Refusing update from remote peer 4a01692d62ce42a9818fda86492823e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:40:33.907971 29724 consensus_queue.cc:1035] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79f691146e6f45018d837e236c65f2e3" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 41569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:40:33.908389 29720 consensus_queue.cc:1035] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f39aec984404eaebec41f56031c112a" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 41893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:40:33.924654 29727 mvcc.cc:205] Tried to move back new op lower bound from 7315641278064525312 to 7315641277895987200. Current Snapshot: MvccSnapshot[applied={T|T < 7315641278064525312}]
I20260806 19:40:33.928357 29729 mvcc.cc:205] Tried to move back new op lower bound from 7315641278064525312 to 7315641277895987200. Current Snapshot: MvccSnapshot[applied={T|T < 7315641278064525312}]
I20260806 19:40:33.951958 29232 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:40:33.952332 29232 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:40:33.956861 29232 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a0f6373f69cb41aab6737d863623ff02: Sending DeleteTablet for 3 replicas of tablet 0eebf031c08d48a99606c6f87cd1a240
I20260806 19:40:33.958531 29626 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eebf031c08d48a99606c6f87cd1a240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:33 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:35904
I20260806 19:40:33.958585 29492 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eebf031c08d48a99606c6f87cd1a240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:33 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:58482
I20260806 19:40:33.958913 29356 tablet_service.cc:1514] Processing DeleteTablet for tablet 0eebf031c08d48a99606c6f87cd1a240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:33 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:56650
I20260806 19:40:33.961405 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 29293
I20260806 19:40:33.972966 29737 tablet_replica.cc:330] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3: stopping tablet replica
I20260806 19:40:33.973594 29737 raft_consensus.cc:2229] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:40:33.974236 29739 tablet_replica.cc:330] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a: stopping tablet replica
I20260806 19:40:33.974467 29737 raft_consensus.cc:2258] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:33.974880 29739 raft_consensus.cc:2229] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:40:33.975490 29739 raft_consensus.cc:2258] T 0eebf031c08d48a99606c6f87cd1a240 P 7f39aec984404eaebec41f56031c112a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:33.976838 29738 tablet_replica.cc:330] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9: stopping tablet replica
I20260806 19:40:33.977458 29738 raft_consensus.cc:2229] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:40:33.978374 29738 raft_consensus.cc:2258] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:33.979126 29737 ts_tablet_manager.cc:1905] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:33.980865 29738 ts_tablet_manager.cc:1905] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 19:40:33 dist-test-slave-ss4n krb5kdc[29187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:40:33.990573 29738 ts_tablet_manager.cc:1918] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:40:33.991216 29738 log.cc:1196] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/0eebf031c08d48a99606c6f87cd1a240
I20260806 19:40:33.991951 29738 ts_tablet_manager.cc:1939] T 0eebf031c08d48a99606c6f87cd1a240 P 4a01692d62ce42a9818fda86492823e9: Deleting consensus metadata
I20260806 19:40:33.994786 29216 catalog_manager.cc:4841] TS 4a01692d62ce42a9818fda86492823e9 (127.10.25.2:45653): tablet 0eebf031c08d48a99606c6f87cd1a240 (table test-table [id=019babad61b74e2187e4c90f5e6ab570]) successfully deleted
I20260806 19:40:33.996482 29737 ts_tablet_manager.cc:1918] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:40:33.996730 29737 log.cc:1196] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/0eebf031c08d48a99606c6f87cd1a240
I20260806 19:40:33.997462 29737 ts_tablet_manager.cc:1939] T 0eebf031c08d48a99606c6f87cd1a240 P 79f691146e6f45018d837e236c65f2e3: Deleting consensus metadata
I20260806 19:40:33.999644 29218 catalog_manager.cc:4841] TS 79f691146e6f45018d837e236c65f2e3 (127.10.25.3:41569): tablet 0eebf031c08d48a99606c6f87cd1a240 (table test-table [id=019babad61b74e2187e4c90f5e6ab570]) successfully deleted
I20260806 19:40:33.999960 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:33.967782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56654 (local address 127.10.25.1:41893)
0806 19:40:33.968451 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:33.968464 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:33.968802 (+ 338us) server_negotiation.cc:407] Connection header received
0806 19:40:33.969908 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:33.969916 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:33.970019 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:33.970107 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:33.976200 (+ 6093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.977626 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.981694 (+ 4068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:33.982529 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:33.982667 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:33.989807 (+ 7140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:33.989832 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:33.989857 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:33.990019 (+ 162us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:33.995432 (+ 5413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:33.996653 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:33.996664 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:33.996668 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:33.996867 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:33.997446 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:33.997453 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:33.997457 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:33.998118 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:33.998473 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:33.999186 (+ 713us) server_negotiation.cc:299] Negotiation successful
0806 19:40:33.999507 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225}
I20260806 19:40:34.002513 29416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:34.122387 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 29293
W20260806 19:40:34.144411 29216 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.25.1:41893 (error 108)
I20260806 19:40:34.144876 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 29427
I20260806 19:40:34.146971 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:34.146438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:41893 (local address 127.0.0.1:56664)
0806 19:40:34.146652 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:34.146787 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:41893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260806 19:40:34.147294 29216 catalog_manager.cc:4584] TS 7f39aec984404eaebec41f56031c112a (127.10.25.1:41893): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0eebf031c08d48a99606c6f87cd1a240: Network error: Client connection negotiation failed: client connection to 127.10.25.1:41893: connect: Connection refused (error 111)
Aug 06 19:40:34 dist-test-slave-ss4n krb5kdc[29187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:40:34.166141 29709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:34.145892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58498 (local address 127.10.25.2:45653)
0806 19:40:34.146105 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:34.146110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:34.146201 (+ 91us) server_negotiation.cc:407] Connection header received
0806 19:40:34.146412 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:34.146418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:34.146522 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:34.146663 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:34.148356 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:34.149315 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:34.151014 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:34.151531 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:34.151709 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:34.157341 (+ 5632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:34.157365 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:34.157392 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:34.157438 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:34.162544 (+ 5106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:34.163630 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:34.163638 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:34.163640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:34.163832 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:34.164297 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:34.164302 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:34.164305 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:34.164750 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:34.165138 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:34.165447 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 19:40:34.165758 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 19:40:34.166942 29552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:34.201179 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:34.200576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:41893 (local address 127.0.0.1:56680)
0806 19:40:34.200869 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:34.200986 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:41893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260806 19:40:34.202538 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:34.202094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:41893 (local address 127.0.0.1:56692)
0806 19:40:34.202292 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:34.202383 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:41893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 19:40:34.263893 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:34.263340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:41893 (local address 127.0.0.1:56700)
0806 19:40:34.263607 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:34.263726 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:41893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260806 19:40:34.265146 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:34.264744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:41893 (local address 127.0.0.1:56704)
0806 19:40:34.264915 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:34.265004 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:41893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 19:40:34.295429 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 29427
I20260806 19:40:34.319077 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 29563
Aug 06 19:40:34 dist-test-slave-ss4n krb5kdc[29187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:40:34.344973 29710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:34.319935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35906 (local address 127.10.25.3:41569)
0806 19:40:34.320177 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:34.320183 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:34.320205 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:40:34.320362 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:34.320366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:34.320443 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:34.320556 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:34.322077 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:34.322831 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:34.324164 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:34.324589 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:34.324734 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:34.336275 (+ 11541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:34.336297 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:34.336316 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:34.336371 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:34.340631 (+ 4260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:34.341743 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:34.341752 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:34.341754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:34.341926 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:34.342822 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:34.342827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:34.342829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:34.343374 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:34.343653 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:34.344368 (+ 715us) server_negotiation.cc:299] Negotiation successful
0806 19:40:34.344619 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 19:40:34.346524 29688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:34.347337 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:34.346514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:41893 (local address 127.0.0.1:56714)
0806 19:40:34.346874 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:34.347047 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:41893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260806 19:40:34.348940 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:34.348271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:41893 (local address 127.0.0.1:56724)
0806 19:40:34.348588 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:34.348723 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:41893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260806 19:40:34.467418 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 29563
I20260806 19:40:34.490175 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 29200
I20260806 19:40:34.491595 29262 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:34.609352 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 29200
2026-08-06T19:40:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5119 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T19:40:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:40:34Z Disabled control of system clock
I20260806 19:40:34.703977 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:44317
--webserver_interface=127.10.25.62
--webserver_port=0
--builtin_ntp_servers=127.10.25.20:36887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:44317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 19:40:35.089825 29749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:35.090158 29749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:35.090251 29749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:35.098582 29749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:40:35.098747 29749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:35.098793 29749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:35.098865 29749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:40:35.098918 29749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:40:35.109761 29749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:36887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:44317
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:44317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:35.111105 29749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:35.113540 29749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:35.126741 29755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:35.126931 29754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:35.126669 29757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:35.127956 29749 server_base.cc:1034] running on GCE node
I20260806 19:40:35.129034 29749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:35.131071 29749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:35.132551 29749 hybrid_clock.cc:648] HybridClock initialized: now 1786045235132441 us; error 72 us; skew 500 ppm
I20260806 19:40:35.133046 29749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:40:35.136070 29749 webserver.cc:460] Webserver started at http://127.10.25.62:34975/ using document root <none> and password file <none>
I20260806 19:40:35.136906 29749 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:35.137020 29749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:35.137374 29749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:35.141217 29749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "c9b312222ba047309b36e3eb002769ad"
format_stamp: "Formatted at 2026-08-06 19:40:35 on dist-test-slave-ss4n"
server_key: "75746451b3292f3527091703748c97a1"
server_key_iv: "511f4ab64d4b49bed14b3230ccc0b7a2"
server_key_version: "encryptionkey@0"
I20260806 19:40:35.142403 29749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "c9b312222ba047309b36e3eb002769ad"
format_stamp: "Formatted at 2026-08-06 19:40:35 on dist-test-slave-ss4n"
server_key: "75746451b3292f3527091703748c97a1"
server_key_iv: "511f4ab64d4b49bed14b3230ccc0b7a2"
server_key_version: "encryptionkey@0"
I20260806 19:40:35.149034 29749 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:40:35.154181 29763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:35.156080 29749 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 19:40:35.156358 29749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "c9b312222ba047309b36e3eb002769ad"
format_stamp: "Formatted at 2026-08-06 19:40:35 on dist-test-slave-ss4n"
server_key: "75746451b3292f3527091703748c97a1"
server_key_iv: "511f4ab64d4b49bed14b3230ccc0b7a2"
server_key_version: "encryptionkey@0"
I20260806 19:40:35.156657 29749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:35.174233 29749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:35.175266 29749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:35.175637 29749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:35.197022 29749 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:44317
I20260806 19:40:35.197046 29815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:44317 every 8 connection(s)
I20260806 19:40:35.198625 29749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:40:35.199779 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 29749
I20260806 19:40:35.199973 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:40:35.200416 10340 external_mini_cluster.cc:1428] Setting key 5f5e4e7b9903051f0d233d295ea6bd8b
I20260806 19:40:35.203928 29816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:35.215590 29819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:35.202654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44076 (local address 127.10.25.62:44317)
0806 19:40:35.203809 (+ 1155us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:35.203821 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:35.203867 (+ 46us) server_negotiation.cc:407] Connection header received
0806 19:40:35.205754 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:35.205779 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:35.206279 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:35.206727 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:35.207395 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:35.208795 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:35.210390 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:35.211132 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:35.211365 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:35.212857 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:35.212865 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:35.212882 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:35.213609 (+ 727us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:35.213702 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:35.213727 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:35.213883 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:35.214159 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 19:40:35.214521 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":419,"threads_started":1}
I20260806 19:40:35.217355 29816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad: Bootstrap starting.
I20260806 19:40:35.222241 29816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:35.223853 29816 log.cc:824] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:35.227422 29816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad: No bootstrap required, opened a new log
I20260806 19:40:35.236444 29816 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b312222ba047309b36e3eb002769ad" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44317 } }
I20260806 19:40:35.236869 29816 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:35.236946 29816 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9b312222ba047309b36e3eb002769ad, State: Initialized, Role: FOLLOWER
I20260806 19:40:35.237654 29816 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b312222ba047309b36e3eb002769ad" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44317 } }
I20260806 19:40:35.237895 29816 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:40:35.238080 29816 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:40:35.238339 29816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:35.240839 29816 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b312222ba047309b36e3eb002769ad" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44317 } }
I20260806 19:40:35.241451 29816 leader_election.cc:302] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9b312222ba047309b36e3eb002769ad; no voters:
I20260806 19:40:35.242241 29816 leader_election.cc:288] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:40:35.242516 29821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:35.243409 29821 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [term 1 LEADER]: Becoming Leader. State: Replica: c9b312222ba047309b36e3eb002769ad, State: Running, Role: LEADER
I20260806 19:40:35.244012 29821 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b312222ba047309b36e3eb002769ad" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44317 } }
I20260806 19:40:35.244611 29816 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:40:35.247881 29823 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9b312222ba047309b36e3eb002769ad. Latest consensus state: current_term: 1 leader_uuid: "c9b312222ba047309b36e3eb002769ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b312222ba047309b36e3eb002769ad" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44317 } } }
I20260806 19:40:35.248463 29823 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:35.250334 29822 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9b312222ba047309b36e3eb002769ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9b312222ba047309b36e3eb002769ad" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 44317 } } }
I20260806 19:40:35.250957 29822 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:35.252772 29826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:40:35.259842 29826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:40:35.272396 29826 catalog_manager.cc:1347] Generated new cluster ID: 52d9a36ee2e64c7688d60c19e21743b9
I20260806 19:40:35.272565 29826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:40:35.286237 29826 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:40:35.288434 29826 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:40:35.297665 29826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad: Generated new TSK 0
I20260806 19:40:35.298691 29826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:40:35.311374 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44317
--builtin_ntp_servers=127.10.25.20:36887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 19:40:35.725188 29840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:35.725613 29840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:35.725796 29840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:35.733850 29840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:35.733989 29840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:35.734179 29840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:40:35.744395 29840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:36887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:35.746001 29840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:35.748003 29840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:35.763978 29846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:35.764025 29849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:35.764161 29847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:35.764979 29840 server_base.cc:1034] running on GCE node
I20260806 19:40:35.765785 29840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:35.766923 29840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:35.768230 29840 hybrid_clock.cc:648] HybridClock initialized: now 1786045235768156 us; error 51 us; skew 500 ppm
I20260806 19:40:35.768709 29840 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:40:35.771824 29840 webserver.cc:460] Webserver started at http://127.10.25.1:33911/ using document root <none> and password file <none>
I20260806 19:40:35.772643 29840 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:35.772796 29840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:35.773149 29840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:35.776510 29840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "12039b23053243a99f71797f727723b2"
format_stamp: "Formatted at 2026-08-06 19:40:35 on dist-test-slave-ss4n"
server_key: "5c74b3cbccee494a767d21753e60ecdf"
server_key_iv: "eec4daa5e215bc39500e19720d5c82b4"
server_key_version: "encryptionkey@0"
I20260806 19:40:35.777459 29840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "12039b23053243a99f71797f727723b2"
format_stamp: "Formatted at 2026-08-06 19:40:35 on dist-test-slave-ss4n"
server_key: "5c74b3cbccee494a767d21753e60ecdf"
server_key_iv: "eec4daa5e215bc39500e19720d5c82b4"
server_key_version: "encryptionkey@0"
I20260806 19:40:35.784246 29840 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260806 19:40:35.788977 29855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:35.790989 29840 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:40:35.791234 29840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "12039b23053243a99f71797f727723b2"
format_stamp: "Formatted at 2026-08-06 19:40:35 on dist-test-slave-ss4n"
server_key: "5c74b3cbccee494a767d21753e60ecdf"
server_key_iv: "eec4daa5e215bc39500e19720d5c82b4"
server_key_version: "encryptionkey@0"
I20260806 19:40:35.791544 29840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:35.827716 29840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:35.828677 29840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:35.828966 29840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:35.830183 29840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:35.831926 29840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:35.832031 29840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:35.832122 29840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:35.832186 29840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:35.878149 29840 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:38851
I20260806 19:40:35.878218 29967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:38851 every 8 connection(s)
I20260806 19:40:35.879899 29840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:40:35.883522 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 29840
I20260806 19:40:35.883931 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:40:35.884605 10340 external_mini_cluster.cc:1428] Setting key 765e99e1e6c463605c570b5f144ac6f5
I20260806 19:40:35.888253 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44317
--builtin_ntp_servers=127.10.25.20:36887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 19:40:35.896447 29970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:35.883052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:59239 (local address 127.10.25.62:44317)
0806 19:40:35.883879 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:35.883891 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:35.885055 (+ 1164us) server_negotiation.cc:407] Connection header received
0806 19:40:35.887583 (+ 2528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:35.887588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:35.887683 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:35.887808 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:35.889246 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:35.890558 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:35.892429 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:35.893197 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:35.893362 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:35.895324 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:35.895336 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:35.895344 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:35.895385 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:35.895446 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:35.895467 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:35.895584 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:35.895836 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 19:40:35.895967 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":332,"threads_started":1}
I20260806 19:40:35.897732 29971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:35.883600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44317 (local address 127.10.25.1:59239)
0806 19:40:35.884834 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:35.884884 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:35.887240 (+ 2356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:35.888129 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:35.888139 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:35.888257 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:35.888997 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:35.889041 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:35.890804 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:35.890808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:35.892234 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:35.892249 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:35.893530 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:35.893539 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:35.893913 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:35.894879 (+ 966us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:40:35.894885 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:35.894944 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:40:35.895057 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:35.895588 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:35.895600 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:35.895611 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 19:40:35.895796 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 19:40:35.896313 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":397,"threads_started":1}
I20260806 19:40:35.899653 29968 heartbeater.cc:347] Connected to a master server at 127.10.25.62:44317
I20260806 19:40:35.900015 29968 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:35.900966 29968 heartbeater.cc:511] Master 127.10.25.62:44317 requested a full tablet report, sending...
I20260806 19:40:35.903587 29781 ts_manager.cc:194] Registered new tserver with Master: 12039b23053243a99f71797f727723b2 (127.10.25.1:38851)
I20260806 19:40:35.905177 29781 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.25.1:59239
W20260806 19:40:36.285676 29972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:36.285991 29972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:36.286147 29972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:36.293954 29972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:36.294114 29972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:36.294518 29972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:40:36.306377 29972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:36887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:36.307901 29972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:36.310266 29972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:36.326640 29977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:36.326741 29978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:36.326768 29981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:36.327962 29972 server_base.cc:1034] running on GCE node
I20260806 19:40:36.328791 29972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:36.330188 29972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:36.331506 29972 hybrid_clock.cc:648] HybridClock initialized: now 1786045236331457 us; error 39 us; skew 500 ppm
I20260806 19:40:36.331974 29972 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:40:36.334985 29972 webserver.cc:460] Webserver started at http://127.10.25.2:46427/ using document root <none> and password file <none>
I20260806 19:40:36.335879 29972 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:36.335992 29972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:36.336303 29972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:36.339970 29972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "8df2d4c46eb640a98d18b40dcb28f950"
format_stamp: "Formatted at 2026-08-06 19:40:36 on dist-test-slave-ss4n"
server_key: "d72d5b72258d05561f7750c4c0ab1eb8"
server_key_iv: "80527360ba5f2bb48ed8c5ae820a06cd"
server_key_version: "encryptionkey@0"
I20260806 19:40:36.340905 29972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "8df2d4c46eb640a98d18b40dcb28f950"
format_stamp: "Formatted at 2026-08-06 19:40:36 on dist-test-slave-ss4n"
server_key: "d72d5b72258d05561f7750c4c0ab1eb8"
server_key_iv: "80527360ba5f2bb48ed8c5ae820a06cd"
server_key_version: "encryptionkey@0"
I20260806 19:40:36.347357 29972 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 19:40:36.351797 29987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:36.353674 29972 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:40:36.353847 29972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "8df2d4c46eb640a98d18b40dcb28f950"
format_stamp: "Formatted at 2026-08-06 19:40:36 on dist-test-slave-ss4n"
server_key: "d72d5b72258d05561f7750c4c0ab1eb8"
server_key_iv: "80527360ba5f2bb48ed8c5ae820a06cd"
server_key_version: "encryptionkey@0"
I20260806 19:40:36.354084 29972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:36.375973 29972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:36.376869 29972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:36.377151 29972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:36.378306 29972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:36.379938 29972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:36.380034 29972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:36.380120 29972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:36.380179 29972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:36.418262 29972 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:41467
I20260806 19:40:36.418447 30100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:41467 every 8 connection(s)
I20260806 19:40:36.419682 29972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:40:36.427388 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 29972
I20260806 19:40:36.427860 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:40:36.428428 10340 external_mini_cluster.cc:1428] Setting key fd0771580fa72f7c355d7aeeea813492
I20260806 19:40:36.432035 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44317
--builtin_ntp_servers=127.10.25.20:36887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 19:40:36.435591 30103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:36.422634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:33513 (local address 127.10.25.62:44317)
0806 19:40:36.423203 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:36.423210 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:36.424499 (+ 1289us) server_negotiation.cc:407] Connection header received
0806 19:40:36.426707 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:36.426712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:36.426793 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:36.426939 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:36.428853 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:36.429958 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:36.432224 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:36.432746 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:36.432906 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:36.434619 (+ 1713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:36.434625 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:36.434629 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:36.434650 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:36.434694 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:36.434708 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:36.434820 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:36.435080 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 19:40:36.435197 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":274,"threads_started":1}
I20260806 19:40:36.436756 30104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:36.423228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44317 (local address 127.10.25.2:33513)
0806 19:40:36.424313 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:36.424360 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:36.426354 (+ 1994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:36.427155 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:36.427164 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:36.427483 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:36.428574 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:36.428613 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:36.430169 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:36.430176 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:36.432041 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:36.432058 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:36.433038 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:36.433048 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:36.433303 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:36.434191 (+ 888us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:40:36.434194 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:36.434241 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:40:36.434342 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:36.434820 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:36.434836 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:36.434849 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 19:40:36.435072 (+ 223us) client_negotiation.cc:240] Negotiation successful
0806 19:40:36.435566 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":367,"threads_started":1}
I20260806 19:40:36.438572 30101 heartbeater.cc:347] Connected to a master server at 127.10.25.62:44317
I20260806 19:40:36.439038 30101 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:36.440253 30101 heartbeater.cc:511] Master 127.10.25.62:44317 requested a full tablet report, sending...
I20260806 19:40:36.442692 29781 ts_manager.cc:194] Registered new tserver with Master: 8df2d4c46eb640a98d18b40dcb28f950 (127.10.25.2:41467)
I20260806 19:40:36.444139 29781 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.25.2:33513
W20260806 19:40:36.831681 30105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:36.832013 30105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:36.832106 30105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:36.839824 30105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:36.840070 30105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:36.840276 30105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:40:36.850113 30105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:36887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:44317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:36.851555 30105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:36.854202 30105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:36.869340 30110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:36.869297 30113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:36.869261 30111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:36.871233 30105 server_base.cc:1034] running on GCE node
I20260806 19:40:36.872012 30105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:36.873172 30105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:36.874476 30105 hybrid_clock.cc:648] HybridClock initialized: now 1786045236874392 us; error 114 us; skew 500 ppm
I20260806 19:40:36.874904 30105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:40:36.881870 30105 webserver.cc:460] Webserver started at http://127.10.25.3:36635/ using document root <none> and password file <none>
I20260806 19:40:36.882942 30105 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:36.883116 30105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:36.883517 30105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:36.887166 30105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "32b7eb8f5f7b43e484da6d7042d0f5d6"
format_stamp: "Formatted at 2026-08-06 19:40:36 on dist-test-slave-ss4n"
server_key: "b355e62dc42c98046734d32a8cfb2d26"
server_key_iv: "774a9cdaf5f5d9f124f6f8610b82df4d"
server_key_version: "encryptionkey@0"
I20260806 19:40:36.888073 30105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "32b7eb8f5f7b43e484da6d7042d0f5d6"
format_stamp: "Formatted at 2026-08-06 19:40:36 on dist-test-slave-ss4n"
server_key: "b355e62dc42c98046734d32a8cfb2d26"
server_key_iv: "774a9cdaf5f5d9f124f6f8610b82df4d"
server_key_version: "encryptionkey@0"
I20260806 19:40:36.894816 30105 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 19:40:36.899355 30119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:36.901149 30105 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 19:40:36.901397 30105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "32b7eb8f5f7b43e484da6d7042d0f5d6"
format_stamp: "Formatted at 2026-08-06 19:40:36 on dist-test-slave-ss4n"
server_key: "b355e62dc42c98046734d32a8cfb2d26"
server_key_iv: "774a9cdaf5f5d9f124f6f8610b82df4d"
server_key_version: "encryptionkey@0"
I20260806 19:40:36.901768 30105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:36.909571 29968 heartbeater.cc:503] Master 127.10.25.62:44317 was elected leader, sending a full tablet report...
I20260806 19:40:36.919589 30105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:36.920558 30105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:36.920845 30105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:36.922137 30105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:36.924014 30105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:36.924127 30105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:36.924310 30105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:36.924419 30105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:36.968353 30105 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:46067
I20260806 19:40:36.968420 30232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:46067 every 8 connection(s)
I20260806 19:40:36.970005 30105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:40:36.973369 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 30105
I20260806 19:40:36.973590 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:40:36.974268 10340 external_mini_cluster.cc:1428] Setting key 997fcc07ee06b22e4d1ef900a6d1070c
I20260806 19:40:36.984732 30235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:36.972946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:39481 (local address 127.10.25.62:44317)
0806 19:40:36.973614 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:36.973621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:36.975009 (+ 1388us) server_negotiation.cc:407] Connection header received
0806 19:40:36.977249 (+ 2240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:36.977255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:36.977354 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:36.977541 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:36.978847 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:36.979699 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:36.981512 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:36.981992 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:36.982149 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:36.983776 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:36.983782 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:36.983785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:36.983807 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:36.983860 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:36.983875 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:36.983987 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:36.984236 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 19:40:36.984351 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":312,"threads_started":1}
I20260806 19:40:36.985978 30236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:36.973562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:44317 (local address 127.10.25.3:39481)
0806 19:40:36.974755 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:36.974810 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:36.976928 (+ 2118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:36.977860 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:36.977874 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:36.978033 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:36.978660 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:36.978679 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:36.979888 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:36.979893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:36.981299 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:36.981313 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:36.982290 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:36.982299 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:36.982611 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:36.983376 (+ 765us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:40:36.983379 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:36.983425 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:40:36.983520 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:36.983988 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:36.984003 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:36.984013 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 19:40:36.984225 (+ 212us) client_negotiation.cc:240] Negotiation successful
0806 19:40:36.984717 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":396,"threads_started":1}
I20260806 19:40:36.988265 30233 heartbeater.cc:347] Connected to a master server at 127.10.25.62:44317
I20260806 19:40:36.988659 30233 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:36.989667 30233 heartbeater.cc:511] Master 127.10.25.62:44317 requested a full tablet report, sending...
I20260806 19:40:36.991709 29781 ts_manager.cc:194] Registered new tserver with Master: 32b7eb8f5f7b43e484da6d7042d0f5d6 (127.10.25.3:46067)
I20260806 19:40:36.993160 29781 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.25.3:39481
I20260806 19:40:37.004706 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:40:37.017002 30235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.009955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44086 (local address 127.10.25.62:44317)
0806 19:40:37.010213 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:37.010220 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:37.010553 (+ 333us) server_negotiation.cc:407] Connection header received
0806 19:40:37.010766 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:37.010771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:37.010844 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:37.010945 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:37.011629 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.012364 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.014077 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.014513 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.014629 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:37.015958 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:37.015963 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:37.015967 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:37.015991 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:37.016031 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:37.016046 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:37.016186 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:37.016585 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 19:40:37.016660 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 19:40:37.022862 29781 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:40:37.031595 29781 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:40:37.064381 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.051970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40154 (local address 127.10.25.1:38851)
0806 19:40:37.053648 (+ 1678us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:37.053660 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:37.053703 (+ 43us) server_negotiation.cc:407] Connection header received
0806 19:40:37.053824 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:37.053832 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:37.054115 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:37.054595 (+ 480us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:37.055140 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.056683 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.060241 (+ 3558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.062201 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.063393 (+ 1192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:37.063687 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:37.063775 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 19:40:37.063978 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1392,"thread_start_us":1220,"threads_started":1}
I20260806 19:40:37.064416 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.052059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40080 (local address 127.10.25.2:41467)
0806 19:40:37.053075 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:37.053088 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:37.053900 (+ 812us) server_negotiation.cc:407] Connection header received
0806 19:40:37.054021 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:37.054029 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:37.054280 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:37.054496 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:37.056415 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.057650 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.060752 (+ 3102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.062226 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.062380 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:37.062668 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:37.063870 (+ 1202us) server_negotiation.cc:299] Negotiation successful
0806 19:40:37.064076 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":803,"thread_start_us":627,"threads_started":1}
I20260806 19:40:37.064817 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.050643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42038 (local address 127.10.25.3:46067)
0806 19:40:37.054147 (+ 3504us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:37.054155 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:37.054190 (+ 35us) server_negotiation.cc:407] Connection header received
0806 19:40:37.054289 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:37.054296 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:37.054528 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:37.054726 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:37.055655 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.057675 (+ 2020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.061324 (+ 3649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.062889 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.063026 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:37.063299 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:37.064003 (+ 704us) server_negotiation.cc:299] Negotiation successful
0806 19:40:37.064156 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3320,"thread_start_us":3142,"threads_started":1}
I20260806 19:40:37.063800 30250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.051253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:38851 (local address 127.0.0.1:40154)
0806 19:40:37.052215 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:37.052240 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:37.052402 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:37.054346 (+ 1944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:37.054353 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:37.054386 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:37.054903 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:37.054919 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.056850 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.056857 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:37.059945 (+ 3088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:37.059963 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.062471 (+ 2508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.062483 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:37.062745 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:37.062904 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 19:40:37.063064 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 19:40:37.063283 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":42,"thread_start_us":2150,"threads_started":1}
I20260806 19:40:37.065405 30247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.050554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:46067 (local address 127.0.0.1:42038)
0806 19:40:37.051433 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:37.051492 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:37.051700 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:37.054916 (+ 3216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:37.054920 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:37.054938 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:37.055450 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:37.055469 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.057866 (+ 2397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.057873 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:37.061094 (+ 3221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:37.061111 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.063431 (+ 2320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.063438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:37.063609 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:37.063645 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 19:40:37.063741 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 19:40:37.063885 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":364,"thread_start_us":400,"threads_started":1}
I20260806 19:40:37.065452 30249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.051427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:41467 (local address 127.0.0.1:40080)
0806 19:40:37.053577 (+ 2150us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:37.053597 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:37.053746 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:37.054774 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:37.054787 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:37.054824 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:37.056180 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:37.056199 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.057796 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.057801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:37.060521 (+ 2720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:37.060535 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.062815 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.062822 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:37.063011 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:37.063045 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 19:40:37.063144 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 19:40:37.063298 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1971,"mutex_wait_us":1140,"thread_start_us":855,"threads_started":1}
I20260806 19:40:37.069178 29903 tablet_service.cc:1467] Processing CreateTablet for tablet 9157d48ad31545e391ebda2cc6645722 (DEFAULT_TABLE table=test-table [id=50d615357ded4f449081ae4456b40783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:37.069317 30035 tablet_service.cc:1467] Processing CreateTablet for tablet 9157d48ad31545e391ebda2cc6645722 (DEFAULT_TABLE table=test-table [id=50d615357ded4f449081ae4456b40783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:37.069317 30168 tablet_service.cc:1467] Processing CreateTablet for tablet 9157d48ad31545e391ebda2cc6645722 (DEFAULT_TABLE table=test-table [id=50d615357ded4f449081ae4456b40783]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:37.071390 30035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9157d48ad31545e391ebda2cc6645722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:37.071383 30168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9157d48ad31545e391ebda2cc6645722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:37.071636 29903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9157d48ad31545e391ebda2cc6645722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:37.085826 30253 tablet_bootstrap.cc:492] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2: Bootstrap starting.
I20260806 19:40:37.089293 30254 tablet_bootstrap.cc:492] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6: Bootstrap starting.
I20260806 19:40:37.090167 30253 tablet_bootstrap.cc:654] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:37.091753 30253 log.cc:824] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:37.093532 30255 tablet_bootstrap.cc:492] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950: Bootstrap starting.
I20260806 19:40:37.094616 30254 tablet_bootstrap.cc:654] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:37.095908 30253 tablet_bootstrap.cc:492] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2: No bootstrap required, opened a new log
I20260806 19:40:37.096148 30254 log.cc:824] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:37.096312 30253 ts_tablet_manager.cc:1397] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 19:40:37.097585 30255 tablet_bootstrap.cc:654] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:37.098923 30255 log.cc:824] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:37.100128 30254 tablet_bootstrap.cc:492] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6: No bootstrap required, opened a new log
I20260806 19:40:37.100468 30254 ts_tablet_manager.cc:1397] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260806 19:40:37.102514 30255 tablet_bootstrap.cc:492] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950: No bootstrap required, opened a new log
I20260806 19:40:37.102816 30255 ts_tablet_manager.cc:1397] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260806 19:40:37.107081 30253 raft_consensus.cc:348] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b7eb8f5f7b43e484da6d7042d0f5d6" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46067 } } peers { permanent_uuid: "8df2d4c46eb640a98d18b40dcb28f950" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41467 } } peers { permanent_uuid: "12039b23053243a99f71797f727723b2" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38851 } }
I20260806 19:40:37.108363 30253 raft_consensus.cc:374] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:37.108501 30253 raft_consensus.cc:729] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12039b23053243a99f71797f727723b2, State: Initialized, Role: FOLLOWER
I20260806 19:40:37.109295 30253 consensus_queue.cc:260] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b7eb8f5f7b43e484da6d7042d0f5d6" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46067 } } peers { permanent_uuid: "8df2d4c46eb640a98d18b40dcb28f950" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41467 } } peers { permanent_uuid: "12039b23053243a99f71797f727723b2" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38851 } }
I20260806 19:40:37.111487 30253 ts_tablet_manager.cc:1428] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260806 19:40:37.112059 30254 raft_consensus.cc:348] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b7eb8f5f7b43e484da6d7042d0f5d6" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46067 } } peers { permanent_uuid: "8df2d4c46eb640a98d18b40dcb28f950" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41467 } } peers { permanent_uuid: "12039b23053243a99f71797f727723b2" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38851 } }
I20260806 19:40:37.112560 30254 raft_consensus.cc:374] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:37.112690 30254 raft_consensus.cc:729] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32b7eb8f5f7b43e484da6d7042d0f5d6, State: Initialized, Role: FOLLOWER
I20260806 19:40:37.113795 30254 consensus_queue.cc:260] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b7eb8f5f7b43e484da6d7042d0f5d6" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46067 } } peers { permanent_uuid: "8df2d4c46eb640a98d18b40dcb28f950" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41467 } } peers { permanent_uuid: "12039b23053243a99f71797f727723b2" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38851 } }
I20260806 19:40:37.115113 30255 raft_consensus.cc:348] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b7eb8f5f7b43e484da6d7042d0f5d6" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46067 } } peers { permanent_uuid: "8df2d4c46eb640a98d18b40dcb28f950" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41467 } } peers { permanent_uuid: "12039b23053243a99f71797f727723b2" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38851 } }
I20260806 19:40:37.115767 30255 raft_consensus.cc:374] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:37.115903 30255 raft_consensus.cc:729] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df2d4c46eb640a98d18b40dcb28f950, State: Initialized, Role: FOLLOWER
I20260806 19:40:37.116562 30233 heartbeater.cc:503] Master 127.10.25.62:44317 was elected leader, sending a full tablet report...
I20260806 19:40:37.116674 30254 ts_tablet_manager.cc:1428] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6: Time spent starting tablet: real 0.016s user 0.012s sys 0.005s
I20260806 19:40:37.117130 30255 consensus_queue.cc:260] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b7eb8f5f7b43e484da6d7042d0f5d6" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46067 } } peers { permanent_uuid: "8df2d4c46eb640a98d18b40dcb28f950" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41467 } } peers { permanent_uuid: "12039b23053243a99f71797f727723b2" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38851 } }
I20260806 19:40:37.118659 30101 heartbeater.cc:503] Master 127.10.25.62:44317 was elected leader, sending a full tablet report...
I20260806 19:40:37.119611 30255 ts_tablet_manager.cc:1428] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
W20260806 19:40:37.133088 29969 tablet.cc:2369] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:40:37.172010 30102 tablet.cc:2369] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:40:37.222121 30234 tablet.cc:2369] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:40:37.321727 30261 raft_consensus.cc:482] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:40:37.322149 30261 raft_consensus.cc:504] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b7eb8f5f7b43e484da6d7042d0f5d6" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46067 } } peers { permanent_uuid: "8df2d4c46eb640a98d18b40dcb28f950" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41467 } } peers { permanent_uuid: "12039b23053243a99f71797f727723b2" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38851 } }
I20260806 19:40:37.324072 30261 leader_election.cc:288] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32b7eb8f5f7b43e484da6d7042d0f5d6 (127.10.25.3:46067), 12039b23053243a99f71797f727723b2 (127.10.25.1:38851)
I20260806 19:40:37.331962 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.324726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:44727 (local address 127.10.25.3:46067)
0806 19:40:37.324968 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:37.324976 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:37.325858 (+ 882us) server_negotiation.cc:407] Connection header received
0806 19:40:37.326143 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:37.326147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:37.326215 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:37.326358 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:37.326977 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.327717 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.330091 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.330944 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.331112 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:37.331233 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:37.331568 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 19:40:37.331712 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 19:40:37.332041 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.324420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:46067 (local address 127.10.25.2:44727)
0806 19:40:37.325684 (+ 1264us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:37.325707 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:37.325900 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:37.326378 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:37.326383 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:37.326412 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:37.326809 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:37.326823 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.327857 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.327865 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:37.329847 (+ 1982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:37.329864 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.331188 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.331195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:37.331383 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:37.331408 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 19:40:37.331535 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 19:40:37.331684 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1101,"thread_start_us":790,"threads_started":1}
I20260806 19:40:37.332183 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.324763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:44669 (local address 127.10.25.1:38851)
0806 19:40:37.324977 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:37.324984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:37.326271 (+ 1287us) server_negotiation.cc:407] Connection header received
0806 19:40:37.326485 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:37.326496 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:37.326576 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:37.326692 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:37.327282 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.328063 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.330229 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.331059 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.331241 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:37.331354 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:37.331639 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 19:40:37.331907 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 19:40:37.332810 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.324551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:38851 (local address 127.10.25.2:44669)
0806 19:40:37.326154 (+ 1603us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:37.326170 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:37.326324 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:37.326711 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:37.326715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:37.326746 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:37.327119 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:37.327128 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.328223 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.328233 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:37.330008 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:37.330023 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.331289 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.331296 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:37.331476 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:37.331499 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 19:40:37.331610 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 19:40:37.331901 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1473,"thread_start_us":692,"threads_started":1}
I20260806 19:40:37.333477 30188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9157d48ad31545e391ebda2cc6645722" candidate_uuid: "8df2d4c46eb640a98d18b40dcb28f950" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32b7eb8f5f7b43e484da6d7042d0f5d6" is_pre_election: true
I20260806 19:40:37.334167 30188 raft_consensus.cc:2454] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df2d4c46eb640a98d18b40dcb28f950 in term 0.
I20260806 19:40:37.334371 29923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9157d48ad31545e391ebda2cc6645722" candidate_uuid: "8df2d4c46eb640a98d18b40dcb28f950" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12039b23053243a99f71797f727723b2" is_pre_election: true
I20260806 19:40:37.334935 29923 raft_consensus.cc:2454] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df2d4c46eb640a98d18b40dcb28f950 in term 0.
I20260806 19:40:37.335055 29988 leader_election.cc:302] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32b7eb8f5f7b43e484da6d7042d0f5d6, 8df2d4c46eb640a98d18b40dcb28f950; no voters:
I20260806 19:40:37.335531 30261 raft_consensus.cc:2781] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:40:37.335819 30261 raft_consensus.cc:482] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:40:37.335975 30261 raft_consensus.cc:3037] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:37.338686 30261 raft_consensus.cc:504] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b7eb8f5f7b43e484da6d7042d0f5d6" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46067 } } peers { permanent_uuid: "8df2d4c46eb640a98d18b40dcb28f950" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41467 } } peers { permanent_uuid: "12039b23053243a99f71797f727723b2" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38851 } }
I20260806 19:40:37.339488 30261 leader_election.cc:288] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [CANDIDATE]: Term 1 election: Requested vote from peers 32b7eb8f5f7b43e484da6d7042d0f5d6 (127.10.25.3:46067), 12039b23053243a99f71797f727723b2 (127.10.25.1:38851)
I20260806 19:40:37.340237 30188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9157d48ad31545e391ebda2cc6645722" candidate_uuid: "8df2d4c46eb640a98d18b40dcb28f950" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32b7eb8f5f7b43e484da6d7042d0f5d6"
I20260806 19:40:37.340480 30188 raft_consensus.cc:3037] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:37.340323 29923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9157d48ad31545e391ebda2cc6645722" candidate_uuid: "8df2d4c46eb640a98d18b40dcb28f950" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12039b23053243a99f71797f727723b2"
I20260806 19:40:37.340718 29923 raft_consensus.cc:3037] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:37.343592 29923 raft_consensus.cc:2454] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df2d4c46eb640a98d18b40dcb28f950 in term 1.
I20260806 19:40:37.343998 30188 raft_consensus.cc:2454] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df2d4c46eb640a98d18b40dcb28f950 in term 1.
I20260806 19:40:37.344264 29989 leader_election.cc:302] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12039b23053243a99f71797f727723b2, 8df2d4c46eb640a98d18b40dcb28f950; no voters:
I20260806 19:40:37.344741 30261 raft_consensus.cc:2781] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:37.345311 30261 raft_consensus.cc:686] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [term 1 LEADER]: Becoming Leader. State: Replica: 8df2d4c46eb640a98d18b40dcb28f950, State: Running, Role: LEADER
I20260806 19:40:37.346007 30261 consensus_queue.cc:237] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b7eb8f5f7b43e484da6d7042d0f5d6" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46067 } } peers { permanent_uuid: "8df2d4c46eb640a98d18b40dcb28f950" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41467 } } peers { permanent_uuid: "12039b23053243a99f71797f727723b2" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38851 } }
I20260806 19:40:37.351382 29780 catalog_manager.cc:5495] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df2d4c46eb640a98d18b40dcb28f950 (127.10.25.2). New cstate: current_term: 1 leader_uuid: "8df2d4c46eb640a98d18b40dcb28f950" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32b7eb8f5f7b43e484da6d7042d0f5d6" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df2d4c46eb640a98d18b40dcb28f950" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 41467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12039b23053243a99f71797f727723b2" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38851 } health_report { overall_health: UNKNOWN } } }
I20260806 19:40:37.442580 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.436434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40088 (local address 127.10.25.2:41467)
0806 19:40:37.436634 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:37.436640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:37.436670 (+ 30us) server_negotiation.cc:407] Connection header received
0806 19:40:37.436784 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:37.436788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:37.436889 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:37.437043 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:40:37.437642 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.438389 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.439901 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.440363 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.440473 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:37.441109 (+ 636us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:40:37.441795 (+ 686us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:40:37.441939 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:37.442283 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 19:40:37.442339 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 19:40:37.453248 30188 raft_consensus.cc:1261] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6 [term 1 FOLLOWER]: Refusing update from remote peer 8df2d4c46eb640a98d18b40dcb28f950: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:40:37.453248 29923 raft_consensus.cc:1261] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2 [term 1 FOLLOWER]: Refusing update from remote peer 8df2d4c46eb640a98d18b40dcb28f950: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:40:37.454614 30265 consensus_queue.cc:1035] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32b7eb8f5f7b43e484da6d7042d0f5d6" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 46067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:40:37.455070 30261 consensus_queue.cc:1035] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12039b23053243a99f71797f727723b2" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:40:37.489394 29781 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:40:37.489877 29781 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:40:37.494652 29781 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9b312222ba047309b36e3eb002769ad: Sending DeleteTablet for 3 replicas of tablet 9157d48ad31545e391ebda2cc6645722
I20260806 19:40:37.496527 30168 tablet_service.cc:1514] Processing DeleteTablet for tablet 9157d48ad31545e391ebda2cc6645722 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:37 UTC) from {username='slave'} at 127.0.0.1:42038
I20260806 19:40:37.496611 29903 tablet_service.cc:1514] Processing DeleteTablet for tablet 9157d48ad31545e391ebda2cc6645722 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:37 UTC) from {username='slave'} at 127.0.0.1:40154
I20260806 19:40:37.496606 30035 tablet_service.cc:1514] Processing DeleteTablet for tablet 9157d48ad31545e391ebda2cc6645722 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:37 UTC) from {username='slave'} at 127.0.0.1:40080
I20260806 19:40:37.503465 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 29840
I20260806 19:40:37.538311 30278 tablet_replica.cc:330] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2: stopping tablet replica
I20260806 19:40:37.539050 30278 raft_consensus.cc:2229] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:40:37.539793 30278 raft_consensus.cc:2258] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:37.540941 30277 tablet_replica.cc:330] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6: stopping tablet replica
I20260806 19:40:37.541501 30277 raft_consensus.cc:2229] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:40:37.542084 30277 raft_consensus.cc:2258] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:37.543241 30278 ts_tablet_manager.cc:1905] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:37.544399 30279 tablet_replica.cc:330] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950: stopping tablet replica
I20260806 19:40:37.544651 30277 ts_tablet_manager.cc:1905] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:37.547263 30279 raft_consensus.cc:2229] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:40:37.548185 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.505680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40164 (local address 127.10.25.1:38851)
0806 19:40:37.505956 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:37.505967 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:37.530959 (+ 24992us) server_negotiation.cc:407] Connection header received
0806 19:40:37.531621 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:37.531632 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:37.531787 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:37.531886 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:37.532761 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.534587 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.542048 (+ 7461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.542843 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.543001 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:37.545651 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:37.545665 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:37.545684 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:37.546557 (+ 873us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:37.546630 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:37.546659 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:37.546764 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:37.547570 (+ 806us) server_negotiation.cc:299] Negotiation successful
0806 19:40:37.547707 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 19:40:37.549875 29963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:37.549753 30279 raft_consensus.cc:2258] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:37.553592 30279 ts_tablet_manager.cc:1905] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:37.554174 30277 ts_tablet_manager.cc:1918] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:40:37.554374 30277 log.cc:1196] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/9157d48ad31545e391ebda2cc6645722
I20260806 19:40:37.555130 30277 ts_tablet_manager.cc:1939] T 9157d48ad31545e391ebda2cc6645722 P 32b7eb8f5f7b43e484da6d7042d0f5d6: Deleting consensus metadata
I20260806 19:40:37.557507 29764 catalog_manager.cc:4841] TS 32b7eb8f5f7b43e484da6d7042d0f5d6 (127.10.25.3:46067): tablet 9157d48ad31545e391ebda2cc6645722 (table test-table [id=50d615357ded4f449081ae4456b40783]) successfully deleted
I20260806 19:40:37.563119 30279 ts_tablet_manager.cc:1918] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:40:37.563354 30279 log.cc:1196] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/9157d48ad31545e391ebda2cc6645722
I20260806 19:40:37.563898 30279 ts_tablet_manager.cc:1939] T 9157d48ad31545e391ebda2cc6645722 P 8df2d4c46eb640a98d18b40dcb28f950: Deleting consensus metadata
I20260806 19:40:37.565989 29767 catalog_manager.cc:4841] TS 8df2d4c46eb640a98d18b40dcb28f950 (127.10.25.2:41467): tablet 9157d48ad31545e391ebda2cc6645722 (table test-table [id=50d615357ded4f449081ae4456b40783]) successfully deleted
I20260806 19:40:37.698531 30278 ts_tablet_manager.cc:1918] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:40:37.698828 30278 log.cc:1196] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/9157d48ad31545e391ebda2cc6645722
I20260806 19:40:37.699630 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 29840
I20260806 19:40:37.699671 30278 ts_tablet_manager.cc:1939] T 9157d48ad31545e391ebda2cc6645722 P 12039b23053243a99f71797f727723b2: Deleting consensus metadata
W20260806 19:40:37.727865 29766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.25.1:38851 (error 108)
I20260806 19:40:37.728814 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 29972
I20260806 19:40:37.730525 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.729360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:38851 (local address 127.0.0.1:40170)
0806 19:40:37.730052 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:37.730221 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":408,"threads_started":1}
W20260806 19:40:37.730932 29766 catalog_manager.cc:4584] TS 12039b23053243a99f71797f727723b2 (127.10.25.1:38851): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9157d48ad31545e391ebda2cc6645722: Network error: Client connection negotiation failed: client connection to 127.10.25.1:38851: connect: Connection refused (error 111)
I20260806 19:40:37.739223 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.730055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40098 (local address 127.10.25.2:41467)
0806 19:40:37.730281 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:37.730290 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:37.730316 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:40:37.730400 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:37.730407 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:37.730548 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:37.730701 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:37.731374 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.732668 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.735151 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.735808 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.735982 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:37.737477 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:37.737544 (+ 67us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:37.737571 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:37.738284 (+ 713us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:37.738347 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:37.738375 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:37.738504 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:37.738683 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 19:40:37.738802 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 19:40:37.740070 30096 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:37.785183 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.784633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:38851 (local address 127.0.0.1:40174)
0806 19:40:37.784915 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:37.785023 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 19:40:37.786597 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.786098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:38851 (local address 127.0.0.1:40178)
0806 19:40:37.786322 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:37.786422 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 19:40:37.847276 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.846805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:38851 (local address 127.0.0.1:40188)
0806 19:40:37.847022 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:37.847110 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 19:40:37.848383 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.848004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:38851 (local address 127.0.0.1:40198)
0806 19:40:37.848158 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:37.848243 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 19:40:37.866245 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 29972
I20260806 19:40:37.888695 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 30105
I20260806 19:40:37.897099 30285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.889780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42042 (local address 127.10.25.3:46067)
0806 19:40:37.890362 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:37.890367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:37.890383 (+ 16us) server_negotiation.cc:407] Connection header received
0806 19:40:37.890442 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:37.890446 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:37.890529 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:37.890676 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:37.891152 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.891932 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.893657 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:37.894182 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:37.894343 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:37.895561 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:37.895570 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:37.895586 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:37.896215 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:37.896267 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:37.896288 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:37.896425 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:37.896604 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 19:40:37.896715 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":308,"threads_started":1}
I20260806 19:40:37.897976 30228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:37.929927 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.929278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:38851 (local address 127.0.0.1:40204)
0806 19:40:37.929644 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:37.929758 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 19:40:37.931139 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:37.930657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:38851 (local address 127.0.0.1:40216)
0806 19:40:37.930875 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:37.930972 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260806 19:40:37.995353 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 30105
I20260806 19:40:38.015260 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 29749
I20260806 19:40:38.016656 29811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:38.130308 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:38.129791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:38851 (local address 127.0.0.1:40228)
0806 19:40:38.129992 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:38.130127 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:38851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 19:40:38.130403 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 29749
2026-08-06T19:40:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3522 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:40:38 dist-test-slave-ss4n krb5kdc[30290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:40:38 dist-test-slave-ss4n krb5kdc[30290](info): set up 2 sockets
Aug 06 19:40:38 dist-test-slave-ss4n krb5kdc[30290](info): commencing operation
krb5kdc: starting...
W20260806 19:40:39.413470 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:40:39 dist-test-slave-ss4n krb5kdc[30290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:40:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:40:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:40:39.767107 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:36443
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:37343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:36443
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:40.140166 30303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:40.140805 30303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:40.141023 30303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:40.149629 30303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:40:40.149793 30303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:40.149871 30303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:40.149984 30303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:40:40.150058 30303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:40:40.160583 30303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:37343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:36443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:36443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:40.162091 30303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:40.163722 30303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:40.176926 30311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:40.177350 30309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:40.177327 30308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:40.179337 30303 server_base.cc:1034] running on GCE node
I20260806 19:40:40.180382 30303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:40.183321 30303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:40.184633 30303 hybrid_clock.cc:648] HybridClock initialized: now 1786045240184574 us; error 44 us; skew 500 ppm
I20260806 19:40:40.185186 30303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:40 dist-test-slave-ss4n krb5kdc[30290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045240, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:40.189453 30303 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:40:40.191661 30303 webserver.cc:460] Webserver started at http://127.10.25.62:43779/ using document root <none> and password file <none>
I20260806 19:40:40.192510 30303 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:40.192698 30303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:40.193071 30303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:40.196789 30303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "59498e0dec5349cabcaf66555f40718c"
format_stamp: "Formatted at 2026-08-06 19:40:40 on dist-test-slave-ss4n"
server_key: "411ec10e5d2afcb4247dea8d162db898"
server_key_iv: "a28e6bc9d4e2f0d80d2f5111f0d61e7e"
server_key_version: "encryptionkey@0"
I20260806 19:40:40.197883 30303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "59498e0dec5349cabcaf66555f40718c"
format_stamp: "Formatted at 2026-08-06 19:40:40 on dist-test-slave-ss4n"
server_key: "411ec10e5d2afcb4247dea8d162db898"
server_key_iv: "a28e6bc9d4e2f0d80d2f5111f0d61e7e"
server_key_version: "encryptionkey@0"
I20260806 19:40:40.204460 30303 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 19:40:40.208694 30318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:40.210487 30303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:40:40.210726 30303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "59498e0dec5349cabcaf66555f40718c"
format_stamp: "Formatted at 2026-08-06 19:40:40 on dist-test-slave-ss4n"
server_key: "411ec10e5d2afcb4247dea8d162db898"
server_key_iv: "a28e6bc9d4e2f0d80d2f5111f0d61e7e"
server_key_version: "encryptionkey@0"
I20260806 19:40:40.210973 30303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:40.246666 30303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:40.251818 30303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:40.252161 30303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:40.274878 30303 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:36443
I20260806 19:40:40.274895 30369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:36443 every 8 connection(s)
I20260806 19:40:40.276750 30303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:40:40.281093 30370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:40.283684 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 30303
I20260806 19:40:40.283900 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:40:40.284474 10340 external_mini_cluster.cc:1428] Setting key 6b34eb247700d69e0e57c0a73c0792b2
I20260806 19:40:40.294588 30370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c: Bootstrap starting.
I20260806 19:40:40.298347 30370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:40.299614 30370 log.cc:824] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c: Log is configured to *not* fsync() on all Append() calls
Aug 06 19:40:40 dist-test-slave-ss4n krb5kdc[30290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:40:40.303203 30370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c: No bootstrap required, opened a new log
I20260806 19:40:40.311266 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:40.286899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46024 (local address 127.10.25.62:36443)
0806 19:40:40.287858 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:40.287872 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:40.287924 (+ 52us) server_negotiation.cc:407] Connection header received
0806 19:40:40.289901 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:40.289929 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:40.290376 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:40.290817 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:40.292554 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:40.294013 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:40.295492 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:40.296246 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:40.296489 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:40.301467 (+ 4978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:40.301621 (+ 154us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:40.301663 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:40.301738 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:40.305837 (+ 4099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:40.306926 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:40.306938 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:40.306948 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:40.307178 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:40.307651 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:40.307655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:40.307658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:40.308311 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:40.308595 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:40.309227 (+ 632us) server_negotiation.cc:299] Negotiation successful
0806 19:40:40.309892 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":417,"threads_started":1}
I20260806 19:40:40.313309 30370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59498e0dec5349cabcaf66555f40718c" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36443 } }
I20260806 19:40:40.313845 30370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:40.313972 30370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59498e0dec5349cabcaf66555f40718c, State: Initialized, Role: FOLLOWER
I20260806 19:40:40.314733 30370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59498e0dec5349cabcaf66555f40718c" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36443 } }
I20260806 19:40:40.315037 30370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:40:40.315210 30370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:40:40.315476 30370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:40.318313 30370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59498e0dec5349cabcaf66555f40718c" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36443 } }
I20260806 19:40:40.318948 30370 leader_election.cc:302] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59498e0dec5349cabcaf66555f40718c; no voters:
I20260806 19:40:40.319916 30370 leader_election.cc:288] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:40:40.320072 30375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:40.321549 30375 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [term 1 LEADER]: Becoming Leader. State: Replica: 59498e0dec5349cabcaf66555f40718c, State: Running, Role: LEADER
I20260806 19:40:40.322147 30375 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59498e0dec5349cabcaf66555f40718c" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36443 } }
I20260806 19:40:40.322576 30370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:40:40.325525 30376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59498e0dec5349cabcaf66555f40718c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59498e0dec5349cabcaf66555f40718c" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36443 } } }
I20260806 19:40:40.326174 30376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:40.325575 30377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59498e0dec5349cabcaf66555f40718c. Latest consensus state: current_term: 1 leader_uuid: "59498e0dec5349cabcaf66555f40718c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59498e0dec5349cabcaf66555f40718c" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 36443 } } }
I20260806 19:40:40.326553 30377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:40.328492 30381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:40:40.335148 30381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:40:40.348415 30381 catalog_manager.cc:1347] Generated new cluster ID: f5e5337f77134f3fb7f760a85bbfdb34
I20260806 19:40:40.348698 30381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:40:40.358668 30381 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:40:40.360146 30381 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:40:40.377404 30381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c: Generated new TSK 0
I20260806 19:40:40.378728 30381 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.25.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
Entry for principal kudu/127.10.25.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab.
I20260806 19:40:40.466490 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36443
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:37343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:40.870440 30396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:40.870796 30396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:40.871001 30396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:40.880600 30396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:40.880806 30396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:40.880970 30396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:40:40.893400 30396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:37343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.1.keytab
--principal=kudu/127.10.25.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36443
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:40.894984 30396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:40.896437 30396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:40.914304 30402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:40.914165 30403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:40.914423 30405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:40.915284 30396 server_base.cc:1034] running on GCE node
I20260806 19:40:40.916134 30396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:40.917281 30396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:40.918612 30396 hybrid_clock.cc:648] HybridClock initialized: now 1786045240918533 us; error 45 us; skew 500 ppm
I20260806 19:40:40.919023 30396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:40 dist-test-slave-ss4n krb5kdc[30290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045240, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:40.923699 30396 init.cc:377] Logged in from keytab as kudu/127.10.25.1@KRBTEST.COM (short username kudu)
I20260806 19:40:40.926651 30396 webserver.cc:460] Webserver started at http://127.10.25.1:41985/ using document root <none> and password file <none>
I20260806 19:40:40.928032 30396 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:40.928301 30396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:40.928727 30396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:40.932438 30396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "755f804fd4904994aa6daa07d288a51f"
format_stamp: "Formatted at 2026-08-06 19:40:40 on dist-test-slave-ss4n"
server_key: "7f8bcef1e9fc0314f5b3adeb1b96d338"
server_key_iv: "d4088759388b0eefe5d1009ee540f5f5"
server_key_version: "encryptionkey@0"
I20260806 19:40:40.933420 30396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "755f804fd4904994aa6daa07d288a51f"
format_stamp: "Formatted at 2026-08-06 19:40:40 on dist-test-slave-ss4n"
server_key: "7f8bcef1e9fc0314f5b3adeb1b96d338"
server_key_iv: "d4088759388b0eefe5d1009ee540f5f5"
server_key_version: "encryptionkey@0"
I20260806 19:40:40.940549 30396 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 19:40:40.945290 30412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:40.947137 30396 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:40:40.947399 30396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "755f804fd4904994aa6daa07d288a51f"
format_stamp: "Formatted at 2026-08-06 19:40:40 on dist-test-slave-ss4n"
server_key: "7f8bcef1e9fc0314f5b3adeb1b96d338"
server_key_iv: "d4088759388b0eefe5d1009ee540f5f5"
server_key_version: "encryptionkey@0"
I20260806 19:40:40.947685 30396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:40.978094 30396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:40.983266 30396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:40.983639 30396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:40.984845 30396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:40.986514 30396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:40.986634 30396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:40.986804 30396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:40.986879 30396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:41.028854 30396 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:38767
I20260806 19:40:41.028908 30527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:38767 every 8 connection(s)
I20260806 19:40:41.030362 30396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:40:41.032780 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 30396
I20260806 19:40:41.033025 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:40:41.033682 10340 external_mini_cluster.cc:1428] Setting key 55a1e4dbc3d6293edf9987c131bcf912
Aug 06 19:40:41 dist-test-slave-ss4n krb5kdc[30290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045240, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.1@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:40:41.064265 30530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:41.034069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:46413 (local address 127.10.25.62:36443)
0806 19:40:41.034674 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:41.034682 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:41.036206 (+ 1524us) server_negotiation.cc:407] Connection header received
0806 19:40:41.039869 (+ 3663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:41.039874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:41.039965 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:41.040091 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:41.042926 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:41.043772 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:41.046843 (+ 3071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:41.047384 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:41.047536 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:41.053596 (+ 6060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:41.053617 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:41.053628 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:41.053711 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:41.059323 (+ 5612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:41.060617 (+ 1294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:41.060629 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:41.060636 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:41.060874 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:41.061533 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:41.061543 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:41.061550 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:41.062073 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:41.062293 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:41.063235 (+ 942us) server_negotiation.cc:299] Negotiation successful
0806 19:40:41.063581 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":309,"threads_started":1}
I20260806 19:40:41.065029 30531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:41.034792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36443 (local address 127.10.25.1:46413)
0806 19:40:41.035988 (+ 1196us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:41.036067 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:41.039408 (+ 3341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:41.040445 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:41.040462 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:41.041541 (+ 1079us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:41.042627 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:41.042666 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:41.044000 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:41.044009 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:41.046502 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:41.046536 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:41.047709 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:41.047720 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:41.048115 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:41.049064 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:40:41.049103 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:41.052873 (+ 3770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:41.059671 (+ 6798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:41.059681 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:41.059699 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:41.060376 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:41.061066 (+ 690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:41.061071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:41.061073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:41.061327 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:41.062283 (+ 956us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:41.062291 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:41.062715 (+ 424us) client_negotiation.cc:769] Sending connection context
0806 19:40:41.063008 (+ 293us) client_negotiation.cc:240] Negotiation successful
0806 19:40:41.063521 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"thread_start_us":482,"threads_started":1}
I20260806 19:40:41.067236 30528 heartbeater.cc:347] Connected to a master server at 127.10.25.62:36443
I20260806 19:40:41.067685 30528 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:41.068900 30528 heartbeater.cc:511] Master 127.10.25.62:36443 requested a full tablet report, sending...
I20260806 19:40:41.072227 30335 ts_manager.cc:194] Registered new tserver with Master: 755f804fd4904994aa6daa07d288a51f (127.10.25.1:38767)
I20260806 19:40:41.073912 30335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.1@KRBTEST.COM'} at 127.10.25.1:46413
WARNING: no policy specified for kudu/127.10.25.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
Entry for principal kudu/127.10.25.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab.
I20260806 19:40:41.118461 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36443
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:37343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:41.504463 30534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:41.504812 30534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:41.505334 30534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:41.519781 30534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:41.519963 30534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:41.520236 30534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:40:41.533454 30534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:37343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.2.keytab
--principal=kudu/127.10.25.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36443
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:41.535653 30534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:41.537226 30534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:41.554216 30539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:41.554884 30540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:41.554905 30542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:41.556212 30534 server_base.cc:1034] running on GCE node
I20260806 19:40:41.557027 30534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:41.558270 30534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:41.559526 30534 hybrid_clock.cc:648] HybridClock initialized: now 1786045241559471 us; error 49 us; skew 500 ppm
I20260806 19:40:41.559916 30534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:41 dist-test-slave-ss4n krb5kdc[30290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045241, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:41.564015 30534 init.cc:377] Logged in from keytab as kudu/127.10.25.2@KRBTEST.COM (short username kudu)
I20260806 19:40:41.566679 30534 webserver.cc:460] Webserver started at http://127.10.25.2:37829/ using document root <none> and password file <none>
I20260806 19:40:41.567477 30534 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:41.567613 30534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:41.567915 30534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:41.571434 30534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "f25cd78d84d24e9f9ce111010810c9ea"
format_stamp: "Formatted at 2026-08-06 19:40:41 on dist-test-slave-ss4n"
server_key: "be16e22f116a1b68f7b313731c999226"
server_key_iv: "59ecd1ba622f09d3ff5d94fbe7b8bbfe"
server_key_version: "encryptionkey@0"
I20260806 19:40:41.572319 30534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "f25cd78d84d24e9f9ce111010810c9ea"
format_stamp: "Formatted at 2026-08-06 19:40:41 on dist-test-slave-ss4n"
server_key: "be16e22f116a1b68f7b313731c999226"
server_key_iv: "59ecd1ba622f09d3ff5d94fbe7b8bbfe"
server_key_version: "encryptionkey@0"
I20260806 19:40:41.580197 30534 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260806 19:40:41.585670 30549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:41.587401 30534 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260806 19:40:41.587580 30534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "f25cd78d84d24e9f9ce111010810c9ea"
format_stamp: "Formatted at 2026-08-06 19:40:41 on dist-test-slave-ss4n"
server_key: "be16e22f116a1b68f7b313731c999226"
server_key_iv: "59ecd1ba622f09d3ff5d94fbe7b8bbfe"
server_key_version: "encryptionkey@0"
I20260806 19:40:41.587792 30534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:41.639099 30534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:41.644670 30534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:41.645164 30534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:41.646425 30534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:41.648048 30534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:41.648137 30534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:41.648229 30534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:41.648276 30534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:41.691886 30534 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:33475
I20260806 19:40:41.691936 30663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:33475 every 8 connection(s)
I20260806 19:40:41.693545 30534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:40:41.697916 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 30534
I20260806 19:40:41.698159 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:40:41.698665 10340 external_mini_cluster.cc:1428] Setting key 943cc8053b403142dd99395936b3b80c
Aug 06 19:40:41 dist-test-slave-ss4n krb5kdc[30290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045241, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.2@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:40:41.723866 30666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:41.696839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:40081 (local address 127.10.25.62:36443)
0806 19:40:41.697367 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:41.697371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:41.698661 (+ 1290us) server_negotiation.cc:407] Connection header received
0806 19:40:41.702526 (+ 3865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:41.702536 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:41.702653 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:41.702805 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:41.704767 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:41.706068 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:41.708398 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:41.709266 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:41.709516 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:41.714074 (+ 4558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:41.714084 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:41.714093 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:41.714155 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:41.717680 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:41.719488 (+ 1808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:41.719494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:41.719497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:41.719729 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:41.720657 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:41.720662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:41.720665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:41.721057 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:41.721226 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:41.722968 (+ 1742us) server_negotiation.cc:299] Negotiation successful
0806 19:40:41.723514 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":298,"threads_started":1}
I20260806 19:40:41.725294 30667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:41.697422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36443 (local address 127.10.25.2:40081)
0806 19:40:41.698446 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:41.698500 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:41.702102 (+ 3602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:41.703077 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:41.703089 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:41.703805 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:41.704506 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:41.704529 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:41.706315 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:41.706320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:41.708163 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:41.708183 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:41.709725 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:41.709733 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:41.710007 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:41.710925 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:40:41.710953 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:41.713685 (+ 2732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:41.718042 (+ 4357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:41.718064 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:41.718100 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:41.719209 (+ 1109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:41.719935 (+ 726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:41.719944 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:41.719951 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:41.720326 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:41.721273 (+ 947us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:41.721288 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:41.722140 (+ 852us) client_negotiation.cc:769] Sending connection context
0806 19:40:41.722756 (+ 616us) client_negotiation.cc:240] Negotiation successful
0806 19:40:41.723511 (+ 755us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":439,"threads_started":1}
I20260806 19:40:41.727393 30664 heartbeater.cc:347] Connected to a master server at 127.10.25.62:36443
I20260806 19:40:41.728000 30664 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:41.729368 30664 heartbeater.cc:511] Master 127.10.25.62:36443 requested a full tablet report, sending...
I20260806 19:40:41.731721 30335 ts_manager.cc:194] Registered new tserver with Master: f25cd78d84d24e9f9ce111010810c9ea (127.10.25.2:33475)
I20260806 19:40:41.732719 30335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.2@KRBTEST.COM'} at 127.10.25.2:40081
WARNING: no policy specified for kudu/127.10.25.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
Entry for principal kudu/127.10.25.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab.
I20260806 19:40:41.783056 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36443
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:37343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:40:42.078421 30528 heartbeater.cc:503] Master 127.10.25.62:36443 was elected leader, sending a full tablet report...
W20260806 19:40:42.176851 30670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:42.177175 30670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:42.177277 30670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:42.185122 30670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:42.185281 30670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:42.185405 30670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:40:42.196388 30670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:37343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.3.keytab
--principal=kudu/127.10.25.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:36443
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:42.197870 30670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:42.199242 30670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:42.214884 30678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:42.214941 30676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:42.214849 30675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:42.215585 30670 server_base.cc:1034] running on GCE node
I20260806 19:40:42.216365 30670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:42.217628 30670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:42.218972 30670 hybrid_clock.cc:648] HybridClock initialized: now 1786045242218862 us; error 74 us; skew 500 ppm
I20260806 19:40:42.219678 30670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:42 dist-test-slave-ss4n krb5kdc[30290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045242, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:42.228966 30670 init.cc:377] Logged in from keytab as kudu/127.10.25.3@KRBTEST.COM (short username kudu)
I20260806 19:40:42.231144 30670 webserver.cc:460] Webserver started at http://127.10.25.3:43667/ using document root <none> and password file <none>
I20260806 19:40:42.232033 30670 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:42.232218 30670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:42.232621 30670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:42.236214 30670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "bf16c2a41b2b4b579a77e31a15d4f7af"
format_stamp: "Formatted at 2026-08-06 19:40:42 on dist-test-slave-ss4n"
server_key: "60b9bd956dc8c4e868b4b7e69cfd179c"
server_key_iv: "bf75c34f4b972860e72d570f1d5b1db7"
server_key_version: "encryptionkey@0"
I20260806 19:40:42.237236 30670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "bf16c2a41b2b4b579a77e31a15d4f7af"
format_stamp: "Formatted at 2026-08-06 19:40:42 on dist-test-slave-ss4n"
server_key: "60b9bd956dc8c4e868b4b7e69cfd179c"
server_key_iv: "bf75c34f4b972860e72d570f1d5b1db7"
server_key_version: "encryptionkey@0"
I20260806 19:40:42.244421 30670 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 19:40:42.249255 30685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:42.251802 30670 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 19:40:42.252086 30670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "bf16c2a41b2b4b579a77e31a15d4f7af"
format_stamp: "Formatted at 2026-08-06 19:40:42 on dist-test-slave-ss4n"
server_key: "60b9bd956dc8c4e868b4b7e69cfd179c"
server_key_iv: "bf75c34f4b972860e72d570f1d5b1db7"
server_key_version: "encryptionkey@0"
I20260806 19:40:42.252790 30670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:42.275247 30670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:42.281870 30670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:42.282249 30670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:42.283414 30670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:42.284981 30670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:42.285110 30670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:42.285275 30670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:42.285353 30670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:42.327864 30670 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:34781
I20260806 19:40:42.328003 30800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:34781 every 8 connection(s)
I20260806 19:40:42.329988 30670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:40:42.335110 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 30670
I20260806 19:40:42.335381 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:40:42.336161 10340 external_mini_cluster.cc:1428] Setting key 4a9397bf47e2eec2429e9dccb6d73db6
Aug 06 19:40:42 dist-test-slave-ss4n krb5kdc[30290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045242, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.3@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:40:42.360172 30803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.333530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:47259 (local address 127.10.25.62:36443)
0806 19:40:42.334117 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:42.334123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:42.335946 (+ 1823us) server_negotiation.cc:407] Connection header received
0806 19:40:42.339586 (+ 3640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:42.339592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:42.339706 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:42.339839 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:42.342381 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.343599 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.345672 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.346173 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.346314 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:42.351196 (+ 4882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:42.351202 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:42.351205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:42.351253 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:42.354947 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:42.356574 (+ 1627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:42.356581 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:42.356584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:42.356848 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:42.357847 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:42.357854 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:42.357878 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:42.358342 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:42.358534 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:42.359550 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0806 19:40:42.359796 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":292,"threads_started":1}
I20260806 19:40:42.361279 30804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.334200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:36443 (local address 127.10.25.3:47259)
0806 19:40:42.335622 (+ 1422us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:42.335710 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:42.339021 (+ 3311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:42.340126 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:42.340136 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:42.341217 (+ 1081us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:42.342060 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:42.342085 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.343825 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.343830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:42.345373 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:42.345390 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.346530 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.346537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:42.346866 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:42.347786 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:40:42.347820 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:42.350831 (+ 3011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:42.355269 (+ 4438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:42.355283 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:42.355345 (+ 62us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:42.356229 (+ 884us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:42.357081 (+ 852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:40:42.357086 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:40:42.357090 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:40:42.357599 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:40:42.358561 (+ 962us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:42.358569 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:42.359066 (+ 497us) client_negotiation.cc:769] Sending connection context
0806 19:40:42.359363 (+ 297us) client_negotiation.cc:240] Negotiation successful
0806 19:40:42.359872 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":525,"threads_started":1}
I20260806 19:40:42.363494 30801 heartbeater.cc:347] Connected to a master server at 127.10.25.62:36443
I20260806 19:40:42.363895 30801 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:42.364804 30801 heartbeater.cc:511] Master 127.10.25.62:36443 requested a full tablet report, sending...
I20260806 19:40:42.366724 30335 ts_manager.cc:194] Registered new tserver with Master: bf16c2a41b2b4b579a77e31a15d4f7af (127.10.25.3:34781)
I20260806 19:40:42.367614 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:40:42.368012 30335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.25.3@KRBTEST.COM'} at 127.10.25.3:47259
I20260806 19:40:42.390987 30803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.373005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56926 (local address 127.10.25.62:36443)
0806 19:40:42.373307 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:42.373314 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:42.373726 (+ 412us) server_negotiation.cc:407] Connection header received
0806 19:40:42.373861 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:42.373868 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:42.373946 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:42.374059 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:42.376667 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.377644 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.379330 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.380029 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.380269 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:42.383409 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:42.383416 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:42.383421 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:42.383481 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:42.387127 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:42.388115 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:42.388121 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:42.388124 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:42.388313 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:42.388849 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:42.388854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:42.388856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:42.389264 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:42.389520 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:42.390053 (+ 533us) server_negotiation.cc:299] Negotiation successful
0806 19:40:42.390274 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260806 19:40:42.396835 30335 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:40:42.405624 30335 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:40:42.439307 30814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.420613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:33475 (local address 127.0.0.1:56642)
0806 19:40:42.424071 (+ 3458us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:42.424139 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:42.425385 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:42.426997 (+ 1612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:42.427011 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:42.427066 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:42.427725 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:42.427748 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.431020 (+ 3272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.431030 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:42.435074 (+ 4044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:42.435107 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.438190 (+ 3083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.438201 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:42.438449 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:42.438506 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 19:40:42.438666 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 19:40:42.438891 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3124,"thread_start_us":2922,"threads_started":1}
I20260806 19:40:42.439375 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.421211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56642 (local address 127.10.25.2:33475)
0806 19:40:42.422628 (+ 1417us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:42.422639 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:42.424264 (+ 1625us) server_negotiation.cc:407] Connection header received
0806 19:40:42.425690 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:42.425706 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:42.426131 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:42.426372 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:42.428096 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.430690 (+ 2594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.435418 (+ 4728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.437613 (+ 2195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.437795 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:42.438076 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:42.438751 (+ 675us) server_negotiation.cc:299] Negotiation successful
0806 19:40:42.438992 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1195,"thread_start_us":1011,"threads_started":1}
I20260806 19:40:42.442539 30816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.421175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:38767 (local address 127.0.0.1:35432)
0806 19:40:42.424672 (+ 3497us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:42.424697 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:42.425081 (+ 384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:42.427946 (+ 2865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:42.427954 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:42.427995 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:42.428582 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:42.428601 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.433568 (+ 4967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.433577 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:42.437829 (+ 4252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:42.437850 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.441421 (+ 3571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.441428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:42.441695 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:42.441750 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 19:40:42.441874 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 19:40:42.442050 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3344,"mutex_wait_us":1,"thread_start_us":3151,"threads_started":1}
I20260806 19:40:42.442791 30817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.422047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35432 (local address 127.10.25.1:38767)
0806 19:40:42.423198 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:42.423208 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:42.424837 (+ 1629us) server_negotiation.cc:407] Connection header received
0806 19:40:42.426933 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:42.426945 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:42.427307 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:42.427587 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:42.430884 (+ 3297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.433206 (+ 2322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.438090 (+ 4884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.441058 (+ 2968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.441335 (+ 277us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:42.441727 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:42.442134 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 19:40:42.442397 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":950,"thread_start_us":753,"threads_started":1}
I20260806 19:40:42.444788 30598 tablet_service.cc:1467] Processing CreateTablet for tablet a8e9d089c7904e0bb77ac53b0d7f31e5 (DEFAULT_TABLE table=test-table [id=5e444f8c36eb453d9fb485bcf5159693]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:42.445863 30818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.423654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:34781 (local address 127.0.0.1:34598)
0806 19:40:42.436123 (+ 12469us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:42.436156 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:42.436309 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:42.437746 (+ 1437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:42.437755 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:42.437791 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:42.438249 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:42.438261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.440514 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.440522 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:42.443232 (+ 2710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:42.443256 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.444827 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.444835 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:42.445034 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:42.445076 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 19:40:42.445232 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 19:40:42.445397 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12287,"thread_start_us":13324,"threads_started":1}
I20260806 19:40:42.446465 30598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8e9d089c7904e0bb77ac53b0d7f31e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:42.446626 30819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.424041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34598 (local address 127.10.25.3:34781)
0806 19:40:42.436975 (+ 12934us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:42.436985 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:42.437022 (+ 37us) server_negotiation.cc:407] Connection header received
0806 19:40:42.437158 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:42.437167 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:42.437456 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:42.437734 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:42.438458 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.440331 (+ 1873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.443449 (+ 3118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.444567 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.444758 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:42.445053 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:42.445299 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 19:40:42.445526 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12725,"thread_start_us":12540,"threads_started":1}
I20260806 19:40:42.449323 30461 tablet_service.cc:1467] Processing CreateTablet for tablet a8e9d089c7904e0bb77ac53b0d7f31e5 (DEFAULT_TABLE table=test-table [id=5e444f8c36eb453d9fb485bcf5159693]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:42.450803 30461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8e9d089c7904e0bb77ac53b0d7f31e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:42.451092 30735 tablet_service.cc:1467] Processing CreateTablet for tablet a8e9d089c7904e0bb77ac53b0d7f31e5 (DEFAULT_TABLE table=test-table [id=5e444f8c36eb453d9fb485bcf5159693]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:42.452903 30735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8e9d089c7904e0bb77ac53b0d7f31e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:42.465811 30821 tablet_bootstrap.cc:492] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea: Bootstrap starting.
I20260806 19:40:42.466777 30820 tablet_bootstrap.cc:492] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f: Bootstrap starting.
I20260806 19:40:42.470507 30821 tablet_bootstrap.cc:654] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:42.471009 30820 tablet_bootstrap.cc:654] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:42.471182 30822 tablet_bootstrap.cc:492] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af: Bootstrap starting.
I20260806 19:40:42.472328 30820 log.cc:824] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:42.472774 30821 log.cc:824] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:42.475653 30822 tablet_bootstrap.cc:654] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:42.476032 30820 tablet_bootstrap.cc:492] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f: No bootstrap required, opened a new log
I20260806 19:40:42.476367 30820 ts_tablet_manager.cc:1397] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 19:40:42.477188 30822 log.cc:824] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:42.478072 30821 tablet_bootstrap.cc:492] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea: No bootstrap required, opened a new log
I20260806 19:40:42.478427 30821 ts_tablet_manager.cc:1397] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260806 19:40:42.481177 30822 tablet_bootstrap.cc:492] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af: No bootstrap required, opened a new log
I20260806 19:40:42.481618 30822 ts_tablet_manager.cc:1397] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260806 19:40:42.484786 30820 raft_consensus.cc:348] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25cd78d84d24e9f9ce111010810c9ea" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33475 } } peers { permanent_uuid: "755f804fd4904994aa6daa07d288a51f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38767 } } peers { permanent_uuid: "bf16c2a41b2b4b579a77e31a15d4f7af" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34781 } }
I20260806 19:40:42.485282 30820 raft_consensus.cc:374] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:42.485386 30820 raft_consensus.cc:729] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 755f804fd4904994aa6daa07d288a51f, State: Initialized, Role: FOLLOWER
I20260806 19:40:42.486162 30820 consensus_queue.cc:260] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25cd78d84d24e9f9ce111010810c9ea" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33475 } } peers { permanent_uuid: "755f804fd4904994aa6daa07d288a51f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38767 } } peers { permanent_uuid: "bf16c2a41b2b4b579a77e31a15d4f7af" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34781 } }
I20260806 19:40:42.488236 30820 ts_tablet_manager.cc:1428] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260806 19:40:42.491263 30821 raft_consensus.cc:348] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25cd78d84d24e9f9ce111010810c9ea" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33475 } } peers { permanent_uuid: "755f804fd4904994aa6daa07d288a51f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38767 } } peers { permanent_uuid: "bf16c2a41b2b4b579a77e31a15d4f7af" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34781 } }
I20260806 19:40:42.491756 30821 raft_consensus.cc:374] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:42.491868 30821 raft_consensus.cc:729] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f25cd78d84d24e9f9ce111010810c9ea, State: Initialized, Role: FOLLOWER
I20260806 19:40:42.492643 30821 consensus_queue.cc:260] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25cd78d84d24e9f9ce111010810c9ea" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33475 } } peers { permanent_uuid: "755f804fd4904994aa6daa07d288a51f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38767 } } peers { permanent_uuid: "bf16c2a41b2b4b579a77e31a15d4f7af" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34781 } }
I20260806 19:40:42.495111 30664 heartbeater.cc:503] Master 127.10.25.62:36443 was elected leader, sending a full tablet report...
I20260806 19:40:42.495234 30821 ts_tablet_manager.cc:1428] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea: Time spent starting tablet: real 0.017s user 0.013s sys 0.003s
I20260806 19:40:42.494844 30822 raft_consensus.cc:348] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25cd78d84d24e9f9ce111010810c9ea" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33475 } } peers { permanent_uuid: "755f804fd4904994aa6daa07d288a51f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38767 } } peers { permanent_uuid: "bf16c2a41b2b4b579a77e31a15d4f7af" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34781 } }
I20260806 19:40:42.495443 30822 raft_consensus.cc:374] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:42.495560 30822 raft_consensus.cc:729] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf16c2a41b2b4b579a77e31a15d4f7af, State: Initialized, Role: FOLLOWER
I20260806 19:40:42.496265 30822 consensus_queue.cc:260] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25cd78d84d24e9f9ce111010810c9ea" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33475 } } peers { permanent_uuid: "755f804fd4904994aa6daa07d288a51f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38767 } } peers { permanent_uuid: "bf16c2a41b2b4b579a77e31a15d4f7af" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34781 } }
I20260806 19:40:42.497701 30801 heartbeater.cc:503] Master 127.10.25.62:36443 was elected leader, sending a full tablet report...
I20260806 19:40:42.498443 30822 ts_tablet_manager.cc:1428] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260806 19:40:42.530498 30826 raft_consensus.cc:482] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:40:42.530974 30826 raft_consensus.cc:504] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25cd78d84d24e9f9ce111010810c9ea" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33475 } } peers { permanent_uuid: "755f804fd4904994aa6daa07d288a51f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38767 } } peers { permanent_uuid: "bf16c2a41b2b4b579a77e31a15d4f7af" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34781 } }
I20260806 19:40:42.533044 30826 leader_election.cc:288] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f25cd78d84d24e9f9ce111010810c9ea (127.10.25.2:33475), bf16c2a41b2b4b579a77e31a15d4f7af (127.10.25.3:34781)
W20260806 19:40:42.533833 30529 tablet.cc:2369] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:40:42.542375 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.533677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:40219 (local address 127.10.25.2:33475)
0806 19:40:42.533945 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:42.533953 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:42.535011 (+ 1058us) server_negotiation.cc:407] Connection header received
0806 19:40:42.535325 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:42.535334 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:42.535445 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:42.535559 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:42.536284 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.537197 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.539644 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.540899 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.541159 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:42.541348 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:42.541828 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 19:40:42.542038 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 19:40:42.542390 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.533517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:33475 (local address 127.10.25.1:40219)
0806 19:40:42.534833 (+ 1316us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:42.534866 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:42.535123 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:42.535595 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:42.535600 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:42.535632 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:42.536100 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:42.536114 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.537368 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.537377 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:42.539429 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:42.539445 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.541277 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.541284 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:42.541557 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:42.541607 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 19:40:42.541732 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 19:40:42.541917 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1078,"mutex_wait_us":1,"thread_start_us":1178,"threads_started":1}
I20260806 19:40:42.542954 30829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.533472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:34781 (local address 127.10.25.1:45897)
0806 19:40:42.535311 (+ 1839us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:42.535331 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:42.535490 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:42.535881 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:42.535885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:42.535907 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:42.536390 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:42.536406 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.538435 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.538440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:42.540158 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:42.540175 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.541937 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.541945 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:42.542182 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:42.542232 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 19:40:42.542369 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 19:40:42.542545 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1258,"thread_start_us":395,"threads_started":1}
I20260806 19:40:42.543283 30819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.533797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:45897 (local address 127.10.25.3:34781)
0806 19:40:42.534034 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:42.534040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:42.535466 (+ 1426us) server_negotiation.cc:407] Connection header received
0806 19:40:42.535616 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:42.535621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:42.535726 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:42.535842 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:42.536724 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.538231 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.540375 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.541559 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.541732 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:42.541852 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:42.542428 (+ 576us) server_negotiation.cc:299] Negotiation successful
0806 19:40:42.542976 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 19:40:42.543735 30618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e9d089c7904e0bb77ac53b0d7f31e5" candidate_uuid: "755f804fd4904994aa6daa07d288a51f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f25cd78d84d24e9f9ce111010810c9ea" is_pre_election: true
I20260806 19:40:42.544448 30618 raft_consensus.cc:2454] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 755f804fd4904994aa6daa07d288a51f in term 0.
I20260806 19:40:42.544358 30755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e9d089c7904e0bb77ac53b0d7f31e5" candidate_uuid: "755f804fd4904994aa6daa07d288a51f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf16c2a41b2b4b579a77e31a15d4f7af" is_pre_election: true
I20260806 19:40:42.544960 30755 raft_consensus.cc:2454] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 755f804fd4904994aa6daa07d288a51f in term 0.
I20260806 19:40:42.545397 30413 leader_election.cc:302] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 755f804fd4904994aa6daa07d288a51f, f25cd78d84d24e9f9ce111010810c9ea; no voters:
I20260806 19:40:42.546002 30826 raft_consensus.cc:2781] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:40:42.546187 30826 raft_consensus.cc:482] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:40:42.546320 30826 raft_consensus.cc:3037] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:42.548885 30826 raft_consensus.cc:504] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25cd78d84d24e9f9ce111010810c9ea" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33475 } } peers { permanent_uuid: "755f804fd4904994aa6daa07d288a51f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38767 } } peers { permanent_uuid: "bf16c2a41b2b4b579a77e31a15d4f7af" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34781 } }
I20260806 19:40:42.550019 30826 leader_election.cc:288] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [CANDIDATE]: Term 1 election: Requested vote from peers f25cd78d84d24e9f9ce111010810c9ea (127.10.25.2:33475), bf16c2a41b2b4b579a77e31a15d4f7af (127.10.25.3:34781)
I20260806 19:40:42.550629 30755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e9d089c7904e0bb77ac53b0d7f31e5" candidate_uuid: "755f804fd4904994aa6daa07d288a51f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf16c2a41b2b4b579a77e31a15d4f7af"
I20260806 19:40:42.550725 30618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e9d089c7904e0bb77ac53b0d7f31e5" candidate_uuid: "755f804fd4904994aa6daa07d288a51f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f25cd78d84d24e9f9ce111010810c9ea"
I20260806 19:40:42.550889 30755 raft_consensus.cc:3037] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:42.551013 30618 raft_consensus.cc:3037] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:42.553871 30618 raft_consensus.cc:2454] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 755f804fd4904994aa6daa07d288a51f in term 1.
I20260806 19:40:42.553954 30755 raft_consensus.cc:2454] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 755f804fd4904994aa6daa07d288a51f in term 1.
I20260806 19:40:42.554445 30413 leader_election.cc:302] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 755f804fd4904994aa6daa07d288a51f, f25cd78d84d24e9f9ce111010810c9ea; no voters:
I20260806 19:40:42.554877 30826 raft_consensus.cc:2781] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:42.555459 30826 raft_consensus.cc:686] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [term 1 LEADER]: Becoming Leader. State: Replica: 755f804fd4904994aa6daa07d288a51f, State: Running, Role: LEADER
I20260806 19:40:42.556104 30826 consensus_queue.cc:237] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25cd78d84d24e9f9ce111010810c9ea" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33475 } } peers { permanent_uuid: "755f804fd4904994aa6daa07d288a51f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38767 } } peers { permanent_uuid: "bf16c2a41b2b4b579a77e31a15d4f7af" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34781 } }
I20260806 19:40:42.562151 30335 catalog_manager.cc:5495] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f reported cstate change: term changed from 0 to 1, leader changed from <none> to 755f804fd4904994aa6daa07d288a51f (127.10.25.1). New cstate: current_term: 1 leader_uuid: "755f804fd4904994aa6daa07d288a51f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25cd78d84d24e9f9ce111010810c9ea" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "755f804fd4904994aa6daa07d288a51f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 38767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf16c2a41b2b4b579a77e31a15d4f7af" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34781 } health_report { overall_health: UNKNOWN } } }
W20260806 19:40:42.582322 30802 tablet.cc:2369] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:40:42.597313 30817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.590643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35448 (local address 127.10.25.1:38767)
0806 19:40:42.591002 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:42.591008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:42.591034 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:40:42.591242 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:42.591246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:42.591356 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:42.591560 (+ 204us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:40:42.592010 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.592882 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.594687 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.595216 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.595376 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:42.595999 (+ 623us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:40:42.596603 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:40:42.596785 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:42.596904 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 19:40:42.597011 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186}
I20260806 19:40:42.608146 30618 raft_consensus.cc:1261] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea [term 1 FOLLOWER]: Refusing update from remote peer 755f804fd4904994aa6daa07d288a51f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:40:42.608146 30755 raft_consensus.cc:1261] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af [term 1 FOLLOWER]: Refusing update from remote peer 755f804fd4904994aa6daa07d288a51f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:40:42.609658 30831 consensus_queue.cc:1035] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f25cd78d84d24e9f9ce111010810c9ea" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:40:42.610107 30826 consensus_queue.cc:1035] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf16c2a41b2b4b579a77e31a15d4f7af" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 34781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:40:42.623554 30834 mvcc.cc:205] Tried to move back new op lower bound from 7315641313707667456 to 7315641313514647552. Current Snapshot: MvccSnapshot[applied={T|T < 7315641313707667456}]
I20260806 19:40:42.643360 30335 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:40:42.643748 30335 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:40:42.648697 30335 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59498e0dec5349cabcaf66555f40718c: Sending DeleteTablet for 3 replicas of tablet a8e9d089c7904e0bb77ac53b0d7f31e5
I20260806 19:40:42.650293 30598 tablet_service.cc:1514] Processing DeleteTablet for tablet a8e9d089c7904e0bb77ac53b0d7f31e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:42 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:56642
I20260806 19:40:42.650293 30461 tablet_service.cc:1514] Processing DeleteTablet for tablet a8e9d089c7904e0bb77ac53b0d7f31e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:42 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:35432
I20260806 19:40:42.650291 30735 tablet_service.cc:1514] Processing DeleteTablet for tablet a8e9d089c7904e0bb77ac53b0d7f31e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:42 UTC) from {username='kudu', principal='kudu/127.10.25.62@KRBTEST.COM'} at 127.0.0.1:34598
I20260806 19:40:42.654624 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 30396
I20260806 19:40:42.662957 30844 tablet_replica.cc:330] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f: stopping tablet replica
I20260806 19:40:42.663957 30844 raft_consensus.cc:2229] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:40:42.665189 30844 raft_consensus.cc:2258] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:42.669072 30844 ts_tablet_manager.cc:1905] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:42.670030 30845 tablet_replica.cc:330] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea: stopping tablet replica
I20260806 19:40:42.670768 30846 tablet_replica.cc:330] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af: stopping tablet replica
I20260806 19:40:42.671347 30846 raft_consensus.cc:2229] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:40:42.671936 30845 raft_consensus.cc:2229] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:40:42.672640 30846 raft_consensus.cc:2258] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:42.673707 30845 raft_consensus.cc:2258] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:42.677649 30846 ts_tablet_manager.cc:1905] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:42.681144 30844 ts_tablet_manager.cc:1918] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 06 19:40:42 dist-test-slave-ss4n krb5kdc[30290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.1@KRBTEST.COM
I20260806 19:40:42.681385 30844 log.cc:1196] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-0/wal/wals/a8e9d089c7904e0bb77ac53b0d7f31e5
I20260806 19:40:42.682140 30844 ts_tablet_manager.cc:1939] T a8e9d089c7904e0bb77ac53b0d7f31e5 P 755f804fd4904994aa6daa07d288a51f: Deleting consensus metadata
I20260806 19:40:42.682412 30845 ts_tablet_manager.cc:1905] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:42.684496 30320 catalog_manager.cc:4841] TS 755f804fd4904994aa6daa07d288a51f (127.10.25.1:38767): tablet a8e9d089c7904e0bb77ac53b0d7f31e5 (table test-table [id=5e444f8c36eb453d9fb485bcf5159693]) successfully deleted
I20260806 19:40:42.687199 30846 ts_tablet_manager.cc:1918] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:40:42.687414 30846 log.cc:1196] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/a8e9d089c7904e0bb77ac53b0d7f31e5
I20260806 19:40:42.688098 30846 ts_tablet_manager.cc:1939] T a8e9d089c7904e0bb77ac53b0d7f31e5 P bf16c2a41b2b4b579a77e31a15d4f7af: Deleting consensus metadata
I20260806 19:40:42.691315 30322 catalog_manager.cc:4841] TS bf16c2a41b2b4b579a77e31a15d4f7af (127.10.25.3:34781): tablet a8e9d089c7904e0bb77ac53b0d7f31e5 (table test-table [id=5e444f8c36eb453d9fb485bcf5159693]) successfully deleted
I20260806 19:40:42.696638 30845 ts_tablet_manager.cc:1918] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:40:42.697072 30845 log.cc:1196] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/a8e9d089c7904e0bb77ac53b0d7f31e5
I20260806 19:40:42.698002 30845 ts_tablet_manager.cc:1939] T a8e9d089c7904e0bb77ac53b0d7f31e5 P f25cd78d84d24e9f9ce111010810c9ea: Deleting consensus metadata
I20260806 19:40:42.698643 30817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.656279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35454 (local address 127.10.25.1:38767)
0806 19:40:42.656636 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:42.656643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:42.659580 (+ 2937us) server_negotiation.cc:407] Connection header received
0806 19:40:42.659868 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:42.659875 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:42.659979 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:42.660124 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:42.669978 (+ 9854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.671295 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.673985 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.674800 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.675298 (+ 498us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:42.686230 (+ 10932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:42.686248 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:42.686279 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:42.686346 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:42.692649 (+ 6303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:42.694504 (+ 1855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:42.694516 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:42.694520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:42.694794 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:42.695405 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:42.695411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:42.695415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:42.695924 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:42.696280 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:42.697882 (+ 1602us) server_negotiation.cc:299] Negotiation successful
0806 19:40:42.698220 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 19:40:42.699738 30523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:42.700642 30319 catalog_manager.cc:4841] TS f25cd78d84d24e9f9ce111010810c9ea (127.10.25.2:33475): tablet a8e9d089c7904e0bb77ac53b0d7f31e5 (table test-table [id=5e444f8c36eb453d9fb485bcf5159693]) successfully deleted
I20260806 19:40:42.818287 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 30396
I20260806 19:40:42.839784 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 30534
Aug 06 19:40:42 dist-test-slave-ss4n krb5kdc[30290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.2@KRBTEST.COM
I20260806 19:40:42.858943 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.840751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56646 (local address 127.10.25.2:33475)
0806 19:40:42.841000 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:42.841005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:42.841027 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:40:42.841237 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:42.841242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:42.841312 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:42.841428 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:42.843518 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.844348 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.846102 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.846527 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.846651 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:42.851780 (+ 5129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:42.851801 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:42.851819 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:42.851871 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:42.855267 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:42.856203 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:42.856212 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:42.856214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:42.856376 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:42.856860 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:42.856865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:42.856867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:42.857457 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:42.857840 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:42.858303 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 19:40:42.858553 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 19:40:42.859751 30659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:42.960901 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 30534
I20260806 19:40:42.984840 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 30670
Aug 06 19:40:42 dist-test-slave-ss4n krb5kdc[30290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.3@KRBTEST.COM
I20260806 19:40:43.010166 30819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:42.986123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34610 (local address 127.10.25.3:34781)
0806 19:40:42.986313 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:42.986317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:42.986341 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:40:42.986582 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:42.986586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:42.986680 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:42.986790 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:42.988472 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.989263 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.991373 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:42.992042 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:42.992162 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:43.002075 (+ 9913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:43.002096 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:43.002134 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:43.002199 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:43.006284 (+ 4085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:43.007446 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:43.007457 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:43.007459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:43.007663 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:43.008322 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:43.008336 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:43.008338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:43.008846 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:43.009128 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:43.009606 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 19:40:43.009821 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 19:40:43.011101 30795 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:43.108780 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 30670
I20260806 19:40:43.132172 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 30303
I20260806 19:40:43.133607 30365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:43.250422 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 30303
2026-08-06T19:40:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5123 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T19:40:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:40:43Z Disabled control of system clock
I20260806 19:40:43.352625 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:45519
--webserver_interface=127.10.25.62
--webserver_port=0
--builtin_ntp_servers=127.10.25.20:40791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:45519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 19:40:43.726929 30855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:43.727293 30855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:43.727382 30855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:43.734956 30855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:40:43.735108 30855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:43.735157 30855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:43.735210 30855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:40:43.735270 30855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:40:43.744987 30855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:40791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:45519
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:45519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:43.746712 30855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:43.749727 30855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:43.764791 30864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:43.764830 30861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:43.764894 30862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:43.766089 30855 server_base.cc:1034] running on GCE node
I20260806 19:40:43.767161 30855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:43.768836 30855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:43.770133 30855 hybrid_clock.cc:648] HybridClock initialized: now 1786045243770068 us; error 88 us; skew 500 ppm
I20260806 19:40:43.770568 30855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:40:43.773720 30855 webserver.cc:460] Webserver started at http://127.10.25.62:42111/ using document root <none> and password file <none>
I20260806 19:40:43.774646 30855 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:43.774832 30855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:43.775249 30855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:43.779086 30855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "d02a4552b1cd4881be549623d34ed7a9"
format_stamp: "Formatted at 2026-08-06 19:40:43 on dist-test-slave-ss4n"
server_key: "7ffb42c62959ca2b428ee4cf1a115e30"
server_key_iv: "b5d5330524dc23abedc1493d6157c76a"
server_key_version: "encryptionkey@0"
I20260806 19:40:43.780241 30855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "d02a4552b1cd4881be549623d34ed7a9"
format_stamp: "Formatted at 2026-08-06 19:40:43 on dist-test-slave-ss4n"
server_key: "7ffb42c62959ca2b428ee4cf1a115e30"
server_key_iv: "b5d5330524dc23abedc1493d6157c76a"
server_key_version: "encryptionkey@0"
I20260806 19:40:43.787050 30855 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 19:40:43.791837 30870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:43.793702 30855 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:40:43.793955 30855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "d02a4552b1cd4881be549623d34ed7a9"
format_stamp: "Formatted at 2026-08-06 19:40:43 on dist-test-slave-ss4n"
server_key: "7ffb42c62959ca2b428ee4cf1a115e30"
server_key_iv: "b5d5330524dc23abedc1493d6157c76a"
server_key_version: "encryptionkey@0"
I20260806 19:40:43.794247 30855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:43.822196 30855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:43.823268 30855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:43.823657 30855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:43.845464 30855 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:45519
I20260806 19:40:43.845553 30922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:45519 every 8 connection(s)
I20260806 19:40:43.847084 30855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:40:43.847679 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 30855
I20260806 19:40:43.847916 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:40:43.848635 10340 external_mini_cluster.cc:1428] Setting key 55d168ec0373e00168a4cee5303b741a
I20260806 19:40:43.852028 30923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:43.863606 30926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:43.850840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36216 (local address 127.10.25.62:45519)
0806 19:40:43.852037 (+ 1197us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:43.852057 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:43.852145 (+ 88us) server_negotiation.cc:407] Connection header received
0806 19:40:43.854025 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:43.854050 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:43.854554 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:43.855079 (+ 525us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:43.855735 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:43.857135 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:43.858679 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:43.859405 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:43.859626 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:43.860977 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:43.860985 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:43.861004 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:43.861730 (+ 726us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:43.861823 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:43.861848 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:43.861982 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:43.862261 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 19:40:43.862607 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":509,"threads_started":1}
I20260806 19:40:43.863929 30923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9: Bootstrap starting.
I20260806 19:40:43.867882 30923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:43.868975 30923 log.cc:824] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:43.872406 30923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9: No bootstrap required, opened a new log
I20260806 19:40:43.882198 30923 raft_consensus.cc:348] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02a4552b1cd4881be549623d34ed7a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 45519 } }
I20260806 19:40:43.882776 30923 raft_consensus.cc:374] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:43.882910 30923 raft_consensus.cc:729] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02a4552b1cd4881be549623d34ed7a9, State: Initialized, Role: FOLLOWER
I20260806 19:40:43.883595 30923 consensus_queue.cc:260] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02a4552b1cd4881be549623d34ed7a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 45519 } }
I20260806 19:40:43.883852 30923 raft_consensus.cc:388] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:40:43.884007 30923 raft_consensus.cc:482] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:40:43.884217 30923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:43.886826 30923 raft_consensus.cc:504] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02a4552b1cd4881be549623d34ed7a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 45519 } }
I20260806 19:40:43.887485 30923 leader_election.cc:302] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d02a4552b1cd4881be549623d34ed7a9; no voters:
I20260806 19:40:43.888252 30923 leader_election.cc:288] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:40:43.888501 30928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:43.889463 30928 raft_consensus.cc:686] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [term 1 LEADER]: Becoming Leader. State: Replica: d02a4552b1cd4881be549623d34ed7a9, State: Running, Role: LEADER
I20260806 19:40:43.890193 30928 consensus_queue.cc:237] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02a4552b1cd4881be549623d34ed7a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 45519 } }
I20260806 19:40:43.890798 30923 sys_catalog.cc:564] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:40:43.893854 30930 sys_catalog.cc:455] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d02a4552b1cd4881be549623d34ed7a9. Latest consensus state: current_term: 1 leader_uuid: "d02a4552b1cd4881be549623d34ed7a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02a4552b1cd4881be549623d34ed7a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 45519 } } }
I20260806 19:40:43.894577 30929 sys_catalog.cc:455] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d02a4552b1cd4881be549623d34ed7a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02a4552b1cd4881be549623d34ed7a9" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 45519 } } }
I20260806 19:40:43.894920 30929 sys_catalog.cc:458] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:43.894766 30930 sys_catalog.cc:458] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9 [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:43.898797 30938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:40:43.904768 30938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:40:43.917972 30938 catalog_manager.cc:1347] Generated new cluster ID: 2bb96782e97a4c6585d76a50c8399282
I20260806 19:40:43.918175 30938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:40:43.947685 30938 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:40:43.949908 30938 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:40:43.963228 30938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9: Generated new TSK 0
I20260806 19:40:43.964596 30938 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:40:43.972223 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.1:0
--local_ip_for_outbound_sockets=127.10.25.1
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:45519
--builtin_ntp_servers=127.10.25.20:40791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 19:40:44.359108 30948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:44.359414 30948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:44.359560 30948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:44.367775 30948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:44.367921 30948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:44.368083 30948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.1
I20260806 19:40:44.378235 30948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:40791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.25.1
--webserver_port=0
--tserver_master_addrs=127.10.25.62:45519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.1
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:44.379698 30948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:44.381883 30948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:44.397815 30955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:44.397835 30957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:44.397940 30954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:44.399739 30948 server_base.cc:1034] running on GCE node
I20260806 19:40:44.400602 30948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:44.401998 30948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:44.403373 30948 hybrid_clock.cc:648] HybridClock initialized: now 1786045244403274 us; error 94 us; skew 500 ppm
I20260806 19:40:44.403826 30948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:40:44.407194 30948 webserver.cc:460] Webserver started at http://127.10.25.1:36787/ using document root <none> and password file <none>
I20260806 19:40:44.408111 30948 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:44.408294 30948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:44.408696 30948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:44.412261 30948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/data/instance:
uuid: "858c2dcf90bb46a4abbe541d20824d8f"
format_stamp: "Formatted at 2026-08-06 19:40:44 on dist-test-slave-ss4n"
server_key: "3dd87694bcb8c22fcc456b9786f41733"
server_key_iv: "a2dfaebd5c883d0e620d9a3755374891"
server_key_version: "encryptionkey@0"
I20260806 19:40:44.413249 30948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance:
uuid: "858c2dcf90bb46a4abbe541d20824d8f"
format_stamp: "Formatted at 2026-08-06 19:40:44 on dist-test-slave-ss4n"
server_key: "3dd87694bcb8c22fcc456b9786f41733"
server_key_iv: "a2dfaebd5c883d0e620d9a3755374891"
server_key_version: "encryptionkey@0"
I20260806 19:40:44.419530 30948 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 19:40:44.424086 30963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:44.425900 30948 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 19:40:44.426159 30948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/wal
uuid: "858c2dcf90bb46a4abbe541d20824d8f"
format_stamp: "Formatted at 2026-08-06 19:40:44 on dist-test-slave-ss4n"
server_key: "3dd87694bcb8c22fcc456b9786f41733"
server_key_iv: "a2dfaebd5c883d0e620d9a3755374891"
server_key_version: "encryptionkey@0"
I20260806 19:40:44.426450 30948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:44.458283 30948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:44.459589 30948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:44.459975 30948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:44.461552 30948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:44.464027 30948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:44.464182 30948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:44.464311 30948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:44.464402 30948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:44.510622 30948 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.1:37463
I20260806 19:40:44.510694 31080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.1:37463 every 8 connection(s)
I20260806 19:40:44.512391 30948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/data/info.pb
I20260806 19:40:44.522845 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 30948
I20260806 19:40:44.523101 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-0/wal/instance
I20260806 19:40:44.523684 10340 external_mini_cluster.cc:1428] Setting key 17f25cbe9692e805e66f41bdacde3d19
I20260806 19:40:44.527156 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.2:0
--local_ip_for_outbound_sockets=127.10.25.2
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:45519
--builtin_ntp_servers=127.10.25.20:40791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 19:40:44.529232 31083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:44.516023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:51245 (local address 127.10.25.62:45519)
0806 19:40:44.516569 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:44.516574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:44.518454 (+ 1880us) server_negotiation.cc:407] Connection header received
0806 19:40:44.520516 (+ 2062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:44.520522 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:44.520625 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:44.520771 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:44.522157 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:44.523040 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:44.525123 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:44.525779 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:44.525916 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:44.527634 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:44.527644 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:44.527652 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:44.527682 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:44.527743 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:44.527764 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:44.527887 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:44.528499 (+ 612us) server_negotiation.cc:299] Negotiation successful
0806 19:40:44.528645 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":282,"threads_started":1}
I20260806 19:40:44.530642 31084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:44.516571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:45519 (local address 127.10.25.1:51245)
0806 19:40:44.518265 (+ 1694us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:44.518305 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:44.520173 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:44.521035 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:44.521045 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:44.521161 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:44.521946 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:44.521967 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:44.523256 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:44.523260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:44.524927 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:44.524947 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:44.526041 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:44.526049 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:44.526289 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:44.527163 (+ 874us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:40:44.527170 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:44.527218 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:40:44.527366 (+ 148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:44.528032 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:44.528053 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:44.528066 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 19:40:44.528458 (+ 392us) client_negotiation.cc:240] Negotiation successful
0806 19:40:44.529023 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"thread_start_us":774,"threads_started":1}
I20260806 19:40:44.532924 31081 heartbeater.cc:347] Connected to a master server at 127.10.25.62:45519
I20260806 19:40:44.533301 31081 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:44.534237 31081 heartbeater.cc:511] Master 127.10.25.62:45519 requested a full tablet report, sending...
I20260806 19:40:44.537390 30888 ts_manager.cc:194] Registered new tserver with Master: 858c2dcf90bb46a4abbe541d20824d8f (127.10.25.1:37463)
I20260806 19:40:44.539113 30888 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.25.1:51245
W20260806 19:40:44.915267 31085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:44.915594 31085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:44.915685 31085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:44.923522 31085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:44.923645 31085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:44.923758 31085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.2
I20260806 19:40:44.934504 31085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:40791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.25.2
--webserver_port=0
--tserver_master_addrs=127.10.25.62:45519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.2
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:44.935931 31085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:44.937892 31085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:44.952767 31091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:44.952853 31092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:44.952824 31094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:44.954450 31085 server_base.cc:1034] running on GCE node
I20260806 19:40:44.955221 31085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:44.956714 31085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:44.958137 31085 hybrid_clock.cc:648] HybridClock initialized: now 1786045244958066 us; error 81 us; skew 500 ppm
I20260806 19:40:44.958840 31085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:40:44.962030 31085 webserver.cc:460] Webserver started at http://127.10.25.2:35809/ using document root <none> and password file <none>
I20260806 19:40:44.962939 31085 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:44.963111 31085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:44.963533 31085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:44.967067 31085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/data/instance:
uuid: "1578720b4b9b40b29242cb634983bbe5"
format_stamp: "Formatted at 2026-08-06 19:40:44 on dist-test-slave-ss4n"
server_key: "7512b6d99c717cb6d1bb1d8ebfcf66f9"
server_key_iv: "833f3324aa13305af852e24a1664a1d4"
server_key_version: "encryptionkey@0"
I20260806 19:40:44.968086 31085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance:
uuid: "1578720b4b9b40b29242cb634983bbe5"
format_stamp: "Formatted at 2026-08-06 19:40:44 on dist-test-slave-ss4n"
server_key: "7512b6d99c717cb6d1bb1d8ebfcf66f9"
server_key_iv: "833f3324aa13305af852e24a1664a1d4"
server_key_version: "encryptionkey@0"
I20260806 19:40:44.974699 31085 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260806 19:40:44.978871 31100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:44.980414 31085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 19:40:44.980599 31085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/wal
uuid: "1578720b4b9b40b29242cb634983bbe5"
format_stamp: "Formatted at 2026-08-06 19:40:44 on dist-test-slave-ss4n"
server_key: "7512b6d99c717cb6d1bb1d8ebfcf66f9"
server_key_iv: "833f3324aa13305af852e24a1664a1d4"
server_key_version: "encryptionkey@0"
I20260806 19:40:44.980880 31085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:45.001282 31085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:45.002293 31085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:45.002627 31085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:45.003760 31085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:45.005466 31085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:45.005602 31085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 19:40:45.005681 31085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:45.005738 31085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:45.046401 31085 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.2:33961
I20260806 19:40:45.046455 31214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.2:33961 every 8 connection(s)
I20260806 19:40:45.047950 31085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/data/info.pb
I20260806 19:40:45.055405 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 31085
I20260806 19:40:45.055680 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/wal/instance
I20260806 19:40:45.056300 10340 external_mini_cluster.cc:1428] Setting key 5f389cf3b65b569cfb9137a495e54cd3
I20260806 19:40:45.060808 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.25.3:0
--local_ip_for_outbound_sockets=127.10.25.3
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:45519
--builtin_ntp_servers=127.10.25.20:40791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 19:40:45.064597 31217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.051116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.2:49407 (local address 127.10.25.62:45519)
0806 19:40:45.051749 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:45.051754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:45.052974 (+ 1220us) server_negotiation.cc:407] Connection header received
0806 19:40:45.055503 (+ 2529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:45.055509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:45.055668 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:45.055832 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:45.057518 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.058441 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.060823 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.061405 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.061627 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.063659 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:45.063665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:45.063669 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:45.063693 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:45.063742 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:45.063757 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:45.063850 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:45.064168 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 19:40:45.064271 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":384,"threads_started":1}
I20260806 19:40:45.066260 31218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.051593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:45519 (local address 127.10.25.2:49407)
0806 19:40:45.052751 (+ 1158us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:45.052800 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:45.055042 (+ 2242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:45.056182 (+ 1140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:45.056198 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:45.056342 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:45.057246 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:45.057271 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.058660 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.058665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:45.060600 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:45.060624 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.061782 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.061793 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:45.062219 (+ 426us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.063146 (+ 927us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:40:45.063152 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:45.063217 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:40:45.063350 (+ 133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:45.063875 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:45.063890 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:45.063904 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 19:40:45.064137 (+ 233us) client_negotiation.cc:240] Negotiation successful
0806 19:40:45.064649 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":355,"threads_started":1}
I20260806 19:40:45.068166 31215 heartbeater.cc:347] Connected to a master server at 127.10.25.62:45519
I20260806 19:40:45.068542 31215 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:45.069415 31215 heartbeater.cc:511] Master 127.10.25.62:45519 requested a full tablet report, sending...
I20260806 19:40:45.071218 30888 ts_manager.cc:194] Registered new tserver with Master: 1578720b4b9b40b29242cb634983bbe5 (127.10.25.2:33961)
I20260806 19:40:45.072170 30888 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.25.2:49407
W20260806 19:40:45.445310 31219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:45.445674 31219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:45.445847 31219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:45.455282 31219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:45.455506 31219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:45.455717 31219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.25.3
I20260806 19:40:45.467911 31219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:40791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.25.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.25.3
--webserver_port=0
--tserver_master_addrs=127.10.25.62:45519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.25.3
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:45.469467 31219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:45.471642 31219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:45.490692 31225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:45.491147 31224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:45.491153 31227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:45.492489 31219 server_base.cc:1034] running on GCE node
I20260806 19:40:45.493290 31219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:45.494544 31219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:45.495862 31219 hybrid_clock.cc:648] HybridClock initialized: now 1786045245495801 us; error 40 us; skew 500 ppm
I20260806 19:40:45.496263 31219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 19:40:45.499423 31219 webserver.cc:460] Webserver started at http://127.10.25.3:42403/ using document root <none> and password file <none>
I20260806 19:40:45.500339 31219 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:45.500494 31219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:45.500897 31219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:45.504415 31219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/data/instance:
uuid: "5045d9f2e90244f59f50fc28dfe63a0e"
format_stamp: "Formatted at 2026-08-06 19:40:45 on dist-test-slave-ss4n"
server_key: "c5f5863eff3c8ebcfde1cc5cb4311bed"
server_key_iv: "19959f1ba89a12ae68085efb07e7feed"
server_key_version: "encryptionkey@0"
I20260806 19:40:45.505393 31219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance:
uuid: "5045d9f2e90244f59f50fc28dfe63a0e"
format_stamp: "Formatted at 2026-08-06 19:40:45 on dist-test-slave-ss4n"
server_key: "c5f5863eff3c8ebcfde1cc5cb4311bed"
server_key_iv: "19959f1ba89a12ae68085efb07e7feed"
server_key_version: "encryptionkey@0"
I20260806 19:40:45.511752 31219 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 19:40:45.516710 31233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:45.518800 31219 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260806 19:40:45.519070 31219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/wal
uuid: "5045d9f2e90244f59f50fc28dfe63a0e"
format_stamp: "Formatted at 2026-08-06 19:40:45 on dist-test-slave-ss4n"
server_key: "c5f5863eff3c8ebcfde1cc5cb4311bed"
server_key_iv: "19959f1ba89a12ae68085efb07e7feed"
server_key_version: "encryptionkey@0"
I20260806 19:40:45.519362 31219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:45.543081 31081 heartbeater.cc:503] Master 127.10.25.62:45519 was elected leader, sending a full tablet report...
I20260806 19:40:45.551834 31219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:45.552809 31219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:45.553139 31219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:45.554421 31219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 19:40:45.556306 31219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 19:40:45.556453 31219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:45.556598 31219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 19:40:45.556691 31219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:45.601016 31219 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.3:43881
I20260806 19:40:45.601073 31346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.3:43881 every 8 connection(s)
I20260806 19:40:45.602519 31219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/data/info.pb
I20260806 19:40:45.609055 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 31219
I20260806 19:40:45.609261 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/wal/instance
I20260806 19:40:45.609879 10340 external_mini_cluster.cc:1428] Setting key efdfac14d516a496d7cbe6769e1b31c7
I20260806 19:40:45.618556 31349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.605843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.3:51417 (local address 127.10.25.62:45519)
0806 19:40:45.606446 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:45.606451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:45.607627 (+ 1176us) server_negotiation.cc:407] Connection header received
0806 19:40:45.610347 (+ 2720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:45.610353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:45.610481 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:45.610643 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:45.612010 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.612935 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.614947 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.615424 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.615596 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.617543 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:45.617552 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:45.617558 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:45.617589 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:45.617635 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:45.617649 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:45.617757 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:45.618026 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 19:40:45.618142 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":346,"threads_started":1}
I20260806 19:40:45.619705 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.606258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:45519 (local address 127.10.25.3:51417)
0806 19:40:45.607415 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:45.607478 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:45.609871 (+ 2393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:45.610962 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:45.610972 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:45.611098 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:40:45.611798 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:45.611818 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.613148 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.613154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:45.614736 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:45.614761 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.615744 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.615754 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:45.616052 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.617059 (+ 1007us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 19:40:45.617063 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:40:45.617115 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 19:40:45.617210 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:40:45.617757 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:40:45.617771 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:40:45.617782 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 19:40:45.617991 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 19:40:45.618469 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":362,"threads_started":1}
I20260806 19:40:45.621425 31347 heartbeater.cc:347] Connected to a master server at 127.10.25.62:45519
I20260806 19:40:45.621853 31347 heartbeater.cc:464] Registering TS with master...
I20260806 19:40:45.622711 31347 heartbeater.cc:511] Master 127.10.25.62:45519 requested a full tablet report, sending...
I20260806 19:40:45.624500 30887 ts_manager.cc:194] Registered new tserver with Master: 5045d9f2e90244f59f50fc28dfe63a0e (127.10.25.3:43881)
I20260806 19:40:45.625778 30887 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.25.3:51417
I20260806 19:40:45.626021 10340 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 19:40:45.637825 31349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.630644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36230 (local address 127.10.25.62:45519)
0806 19:40:45.630885 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:45.630890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:45.631385 (+ 495us) server_negotiation.cc:407] Connection header received
0806 19:40:45.631522 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:45.631526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:45.631604 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:45.631695 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:45.632357 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.633098 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.634734 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.635226 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.635353 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.636953 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:45.636958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:45.636961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:45.636982 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:45.637017 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:45.637029 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:45.637107 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:45.637320 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 19:40:45.637459 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 19:40:45.642853 30887 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 19:40:45.650319 30887 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 19:40:45.681113 31360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.665595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:37463 (local address 127.0.0.1:55888)
0806 19:40:45.666420 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:45.666460 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:45.666713 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:45.668687 (+ 1974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:45.668700 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:45.668746 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:45.669283 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:45.669301 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.673678 (+ 4377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.673690 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:45.677290 (+ 3600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:45.677336 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.680018 (+ 2682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.680027 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:45.680339 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.680408 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 19:40:45.680524 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 19:40:45.680746 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":356,"threads_started":1}
I20260806 19:40:45.682498 31361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.665939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55888 (local address 127.10.25.1:37463)
0806 19:40:45.667813 (+ 1874us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:45.667823 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:45.667856 (+ 33us) server_negotiation.cc:407] Connection header received
0806 19:40:45.667986 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:45.667998 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:45.668341 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:45.669633 (+ 1292us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:45.669927 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.672674 (+ 2747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.677803 (+ 5129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.679764 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.681258 (+ 1494us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.681667 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:45.681764 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 19:40:45.681997 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1675,"thread_start_us":1503,"threads_started":1}
I20260806 19:40:45.687532 31013 tablet_service.cc:1467] Processing CreateTablet for tablet 7767d880bcbf487785e7161fd03e5cf3 (DEFAULT_TABLE table=test-table [id=c03eca4005e24b73bcca5dda47739389]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:45.689611 31013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7767d880bcbf487785e7161fd03e5cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:45.690047 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.666926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:33961 (local address 127.0.0.1:52696)
0806 19:40:45.679098 (+ 12172us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:45.679124 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:45.679275 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:45.681231 (+ 1956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:45.681239 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:45.681275 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:45.681725 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:45.681740 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.683884 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.683890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:45.687109 (+ 3219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:45.687129 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.689116 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.689123 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:45.689325 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.689366 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 19:40:45.689476 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 19:40:45.689678 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11986,"mutex_wait_us":99,"thread_start_us":11825,"threads_started":1}
I20260806 19:40:45.690075 31364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.667690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52696 (local address 127.10.25.2:33961)
0806 19:40:45.680522 (+ 12832us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:45.680533 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:45.680565 (+ 32us) server_negotiation.cc:407] Connection header received
0806 19:40:45.680694 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:45.680702 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:45.680972 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:45.681960 (+ 988us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:45.682185 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.683690 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.687377 (+ 3687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.688850 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.689023 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.689288 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:45.689584 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 19:40:45.689756 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12653,"thread_start_us":12531,"threads_started":1}
I20260806 19:40:45.691372 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.667044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:43881 (local address 127.0.0.1:49096)
0806 19:40:45.679850 (+ 12806us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:45.679871 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:45.679996 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:45.681793 (+ 1797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:45.681798 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:45.681827 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:45.682269 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:45.682287 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.684173 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.684180 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:45.686551 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:45.686571 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.690510 (+ 3939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.690523 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:45.690711 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.690747 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 19:40:45.690823 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 19:40:45.690967 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12675,"mutex_wait_us":68,"thread_start_us":12522,"threads_started":1}
I20260806 19:40:45.692395 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.667704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49096 (local address 127.10.25.3:43881)
0806 19:40:45.673475 (+ 5771us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:45.673509 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:45.680787 (+ 7278us) server_negotiation.cc:407] Connection header received
0806 19:40:45.680906 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:45.680914 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:45.681205 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:45.681419 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:45.682520 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.684021 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.686851 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.688308 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.688460 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.691844 (+ 3384us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:45.691928 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 19:40:45.692062 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5536,"thread_start_us":5353,"threads_started":1}
I20260806 19:40:45.694701 31148 tablet_service.cc:1467] Processing CreateTablet for tablet 7767d880bcbf487785e7161fd03e5cf3 (DEFAULT_TABLE table=test-table [id=c03eca4005e24b73bcca5dda47739389]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:45.695715 31281 tablet_service.cc:1467] Processing CreateTablet for tablet 7767d880bcbf487785e7161fd03e5cf3 (DEFAULT_TABLE table=test-table [id=c03eca4005e24b73bcca5dda47739389]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 19:40:45.696836 31148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7767d880bcbf487785e7161fd03e5cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:45.697552 31281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7767d880bcbf487785e7161fd03e5cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:45.706713 31366 tablet_bootstrap.cc:492] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f: Bootstrap starting.
I20260806 19:40:45.711442 31366 tablet_bootstrap.cc:654] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:45.712930 31366 log.cc:824] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:45.713265 31367 tablet_bootstrap.cc:492] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5: Bootstrap starting.
I20260806 19:40:45.716776 31366 tablet_bootstrap.cc:492] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f: No bootstrap required, opened a new log
I20260806 19:40:45.717129 31366 ts_tablet_manager.cc:1397] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 19:40:45.717233 31367 tablet_bootstrap.cc:654] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:45.718045 31368 tablet_bootstrap.cc:492] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e: Bootstrap starting.
I20260806 19:40:45.718684 31367 log.cc:824] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:45.721868 31367 tablet_bootstrap.cc:492] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5: No bootstrap required, opened a new log
I20260806 19:40:45.722080 31368 tablet_bootstrap.cc:654] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:45.722282 31367 ts_tablet_manager.cc:1397] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260806 19:40:45.723522 31368 log.cc:824] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:45.728320 31368 tablet_bootstrap.cc:492] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e: No bootstrap required, opened a new log
I20260806 19:40:45.728710 31368 ts_tablet_manager.cc:1397] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 19:40:45.729862 31366 raft_consensus.cc:348] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858c2dcf90bb46a4abbe541d20824d8f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37463 } } peers { permanent_uuid: "5045d9f2e90244f59f50fc28dfe63a0e" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43881 } } peers { permanent_uuid: "1578720b4b9b40b29242cb634983bbe5" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33961 } }
I20260806 19:40:45.730666 31366 raft_consensus.cc:374] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:45.730855 31366 raft_consensus.cc:729] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 858c2dcf90bb46a4abbe541d20824d8f, State: Initialized, Role: FOLLOWER
I20260806 19:40:45.731985 31366 consensus_queue.cc:260] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858c2dcf90bb46a4abbe541d20824d8f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37463 } } peers { permanent_uuid: "5045d9f2e90244f59f50fc28dfe63a0e" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43881 } } peers { permanent_uuid: "1578720b4b9b40b29242cb634983bbe5" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33961 } }
I20260806 19:40:45.734790 31366 ts_tablet_manager.cc:1428] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f: Time spent starting tablet: real 0.018s user 0.018s sys 0.000s
I20260806 19:40:45.734622 31367 raft_consensus.cc:348] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858c2dcf90bb46a4abbe541d20824d8f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37463 } } peers { permanent_uuid: "5045d9f2e90244f59f50fc28dfe63a0e" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43881 } } peers { permanent_uuid: "1578720b4b9b40b29242cb634983bbe5" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33961 } }
I20260806 19:40:45.735239 31367 raft_consensus.cc:374] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:45.735383 31367 raft_consensus.cc:729] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1578720b4b9b40b29242cb634983bbe5, State: Initialized, Role: FOLLOWER
I20260806 19:40:45.736243 31367 consensus_queue.cc:260] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858c2dcf90bb46a4abbe541d20824d8f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37463 } } peers { permanent_uuid: "5045d9f2e90244f59f50fc28dfe63a0e" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43881 } } peers { permanent_uuid: "1578720b4b9b40b29242cb634983bbe5" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33961 } }
I20260806 19:40:45.740010 31215 heartbeater.cc:503] Master 127.10.25.62:45519 was elected leader, sending a full tablet report...
I20260806 19:40:45.740950 31367 ts_tablet_manager.cc:1428] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5: Time spent starting tablet: real 0.019s user 0.014s sys 0.004s
I20260806 19:40:45.742527 31368 raft_consensus.cc:348] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858c2dcf90bb46a4abbe541d20824d8f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37463 } } peers { permanent_uuid: "5045d9f2e90244f59f50fc28dfe63a0e" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43881 } } peers { permanent_uuid: "1578720b4b9b40b29242cb634983bbe5" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33961 } }
I20260806 19:40:45.743157 31368 raft_consensus.cc:374] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:45.743273 31368 raft_consensus.cc:729] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5045d9f2e90244f59f50fc28dfe63a0e, State: Initialized, Role: FOLLOWER
I20260806 19:40:45.744463 31368 consensus_queue.cc:260] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858c2dcf90bb46a4abbe541d20824d8f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37463 } } peers { permanent_uuid: "5045d9f2e90244f59f50fc28dfe63a0e" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43881 } } peers { permanent_uuid: "1578720b4b9b40b29242cb634983bbe5" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33961 } }
I20260806 19:40:45.746261 31347 heartbeater.cc:503] Master 127.10.25.62:45519 was elected leader, sending a full tablet report...
I20260806 19:40:45.747419 31368 ts_tablet_manager.cc:1428] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e: Time spent starting tablet: real 0.019s user 0.014s sys 0.004s
W20260806 19:40:45.766081 31082 tablet.cc:2369] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 19:40:45.800582 31216 tablet.cc:2369] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:40:45.810447 31372 raft_consensus.cc:482] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 19:40:45.810765 31372 raft_consensus.cc:504] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858c2dcf90bb46a4abbe541d20824d8f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37463 } } peers { permanent_uuid: "5045d9f2e90244f59f50fc28dfe63a0e" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43881 } } peers { permanent_uuid: "1578720b4b9b40b29242cb634983bbe5" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33961 } }
I20260806 19:40:45.812709 31372 leader_election.cc:288] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5045d9f2e90244f59f50fc28dfe63a0e (127.10.25.3:43881), 1578720b4b9b40b29242cb634983bbe5 (127.10.25.2:33961)
I20260806 19:40:45.820573 31364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.814475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:54111 (local address 127.10.25.2:33961)
0806 19:40:45.814718 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:45.814723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:45.814768 (+ 45us) server_negotiation.cc:407] Connection header received
0806 19:40:45.814851 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:45.814857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:45.814962 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:45.815064 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:45.815641 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.816361 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.818621 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.819397 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.819571 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.819680 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:45.819986 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 19:40:45.820123 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 19:40:45.820721 31375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.813286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.2:33961 (local address 127.10.25.1:54111)
0806 19:40:45.814477 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:45.814495 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:45.814666 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:45.815122 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:45.815126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:45.815149 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:45.815464 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:45.815473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.816616 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.816623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:45.818367 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:45.818386 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.819635 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.819642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:45.819806 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.819831 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 19:40:45.819945 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 19:40:45.820093 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1052,"mutex_wait_us":52,"thread_start_us":592,"threads_started":1}
I20260806 19:40:45.820982 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.813227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.25.1:44153 (local address 127.10.25.3:43881)
0806 19:40:45.813553 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:45.813560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:45.814375 (+ 815us) server_negotiation.cc:407] Connection header received
0806 19:40:45.815348 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:45.815354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:45.815435 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:45.815553 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 19:40:45.816132 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.816903 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.819040 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.819951 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.820143 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.820255 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:45.820568 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 19:40:45.820710 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 19:40:45.821142 31376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.812948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.3:43881 (local address 127.10.25.1:44153)
0806 19:40:45.814238 (+ 1290us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:45.814274 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 19:40:45.814518 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:40:45.815583 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:40:45.815588 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:40:45.815617 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 19:40:45.815960 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:45.815969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.817085 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.817096 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:45.818822 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:40:45.818836 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.820188 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.820196 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:40:45.820377 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.820404 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 19:40:45.820518 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 19:40:45.820682 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1115,"mutex_wait_us":55,"thread_start_us":592,"threads_started":1}
I20260806 19:40:45.822243 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7767d880bcbf487785e7161fd03e5cf3" candidate_uuid: "858c2dcf90bb46a4abbe541d20824d8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1578720b4b9b40b29242cb634983bbe5" is_pre_election: true
I20260806 19:40:45.822244 31301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7767d880bcbf487785e7161fd03e5cf3" candidate_uuid: "858c2dcf90bb46a4abbe541d20824d8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5045d9f2e90244f59f50fc28dfe63a0e" is_pre_election: true
I20260806 19:40:45.822809 31169 raft_consensus.cc:2454] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 858c2dcf90bb46a4abbe541d20824d8f in term 0.
I20260806 19:40:45.822829 31301 raft_consensus.cc:2454] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 858c2dcf90bb46a4abbe541d20824d8f in term 0.
I20260806 19:40:45.823580 30964 leader_election.cc:302] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1578720b4b9b40b29242cb634983bbe5, 5045d9f2e90244f59f50fc28dfe63a0e, 858c2dcf90bb46a4abbe541d20824d8f; no voters:
I20260806 19:40:45.824074 31372 raft_consensus.cc:2781] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 19:40:45.824226 31372 raft_consensus.cc:482] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 19:40:45.824312 31372 raft_consensus.cc:3037] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:45.827194 31372 raft_consensus.cc:504] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858c2dcf90bb46a4abbe541d20824d8f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37463 } } peers { permanent_uuid: "5045d9f2e90244f59f50fc28dfe63a0e" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43881 } } peers { permanent_uuid: "1578720b4b9b40b29242cb634983bbe5" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33961 } }
I20260806 19:40:45.828122 31372 leader_election.cc:288] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [CANDIDATE]: Term 1 election: Requested vote from peers 5045d9f2e90244f59f50fc28dfe63a0e (127.10.25.3:43881), 1578720b4b9b40b29242cb634983bbe5 (127.10.25.2:33961)
I20260806 19:40:45.828845 31301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7767d880bcbf487785e7161fd03e5cf3" candidate_uuid: "858c2dcf90bb46a4abbe541d20824d8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5045d9f2e90244f59f50fc28dfe63a0e"
I20260806 19:40:45.829109 31301 raft_consensus.cc:3037] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:45.829232 31169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7767d880bcbf487785e7161fd03e5cf3" candidate_uuid: "858c2dcf90bb46a4abbe541d20824d8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1578720b4b9b40b29242cb634983bbe5"
I20260806 19:40:45.829672 31169 raft_consensus.cc:3037] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:45.832793 31301 raft_consensus.cc:2454] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 858c2dcf90bb46a4abbe541d20824d8f in term 1.
I20260806 19:40:45.833547 30966 leader_election.cc:302] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5045d9f2e90244f59f50fc28dfe63a0e, 858c2dcf90bb46a4abbe541d20824d8f; no voters:
I20260806 19:40:45.833777 31169 raft_consensus.cc:2454] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 858c2dcf90bb46a4abbe541d20824d8f in term 1.
I20260806 19:40:45.834064 31372 raft_consensus.cc:2781] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:45.834720 31372 raft_consensus.cc:686] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [term 1 LEADER]: Becoming Leader. State: Replica: 858c2dcf90bb46a4abbe541d20824d8f, State: Running, Role: LEADER
I20260806 19:40:45.835274 31372 consensus_queue.cc:237] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858c2dcf90bb46a4abbe541d20824d8f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37463 } } peers { permanent_uuid: "5045d9f2e90244f59f50fc28dfe63a0e" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43881 } } peers { permanent_uuid: "1578720b4b9b40b29242cb634983bbe5" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33961 } }
I20260806 19:40:45.840947 30887 catalog_manager.cc:5495] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 858c2dcf90bb46a4abbe541d20824d8f (127.10.25.1). New cstate: current_term: 1 leader_uuid: "858c2dcf90bb46a4abbe541d20824d8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "858c2dcf90bb46a4abbe541d20824d8f" member_type: VOTER last_known_addr { host: "127.10.25.1" port: 37463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5045d9f2e90244f59f50fc28dfe63a0e" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1578720b4b9b40b29242cb634983bbe5" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33961 } health_report { overall_health: UNKNOWN } } }
W20260806 19:40:45.854624 31348 tablet.cc:2369] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:40:45.901011 31361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.894338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55896 (local address 127.10.25.1:37463)
0806 19:40:45.894710 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:45.894716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:45.894737 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:40:45.894877 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:45.894881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:45.895002 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:45.895228 (+ 226us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:40:45.895897 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.896755 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.898481 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.898988 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.899163 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.899804 (+ 641us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:40:45.900356 (+ 552us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:40:45.900533 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:45.900647 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 19:40:45.900741 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260806 19:40:45.912994 31169 raft_consensus.cc:1261] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5 [term 1 FOLLOWER]: Refusing update from remote peer 858c2dcf90bb46a4abbe541d20824d8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:40:45.913460 31301 raft_consensus.cc:1261] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e [term 1 FOLLOWER]: Refusing update from remote peer 858c2dcf90bb46a4abbe541d20824d8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:40:45.914959 31377 consensus_queue.cc:1035] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1578720b4b9b40b29242cb634983bbe5" member_type: VOTER last_known_addr { host: "127.10.25.2" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:40:45.915638 31372 consensus_queue.cc:1035] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5045d9f2e90244f59f50fc28dfe63a0e" member_type: VOTER last_known_addr { host: "127.10.25.3" port: 43881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:40:45.935307 31380 mvcc.cc:205] Tried to move back new op lower bound from 7315641327244775424 to 7315641326945280000. Current Snapshot: MvccSnapshot[applied={T|T < 7315641327244775424}]
I20260806 19:40:45.953835 30888 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:40:45.954145 30888 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 19:40:45.958958 30888 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d02a4552b1cd4881be549623d34ed7a9: Sending DeleteTablet for 3 replicas of tablet 7767d880bcbf487785e7161fd03e5cf3
I20260806 19:40:45.961098 31013 tablet_service.cc:1514] Processing DeleteTablet for tablet 7767d880bcbf487785e7161fd03e5cf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:45 UTC) from {username='slave'} at 127.0.0.1:55888
I20260806 19:40:45.961357 31281 tablet_service.cc:1514] Processing DeleteTablet for tablet 7767d880bcbf487785e7161fd03e5cf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:45 UTC) from {username='slave'} at 127.0.0.1:49096
I20260806 19:40:45.961848 31148 tablet_service.cc:1514] Processing DeleteTablet for tablet 7767d880bcbf487785e7161fd03e5cf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 19:40:45 UTC) from {username='slave'} at 127.0.0.1:52696
I20260806 19:40:45.968740 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 30948
I20260806 19:40:45.980512 31389 tablet_replica.cc:330] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e: stopping tablet replica
I20260806 19:40:45.981242 31389 raft_consensus.cc:2229] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:40:45.982079 31389 raft_consensus.cc:2258] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:45.985040 31389 ts_tablet_manager.cc:1905] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:45.986200 31391 tablet_replica.cc:330] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f: stopping tablet replica
I20260806 19:40:45.986881 31391 raft_consensus.cc:2229] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [term 1 LEADER]: Raft consensus shutting down.
I20260806 19:40:45.987624 31391 raft_consensus.cc:2258] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:45.989133 31361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:45.970044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55902 (local address 127.10.25.1:37463)
0806 19:40:45.970277 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:45.970284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:45.979407 (+ 9123us) server_negotiation.cc:407] Connection header received
0806 19:40:45.979657 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:45.979663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:45.979805 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:45.979952 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:45.980710 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.982029 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.984066 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:45.984831 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:45.984969 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:45.986745 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:45.986763 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:45.986781 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:45.987652 (+ 871us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:45.987726 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:45.987754 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:45.987845 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:45.988529 (+ 684us) server_negotiation.cc:299] Negotiation successful
0806 19:40:45.988679 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 19:40:45.989876 31076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:45.990406 31390 tablet_replica.cc:330] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5: stopping tablet replica
I20260806 19:40:45.991066 31390 raft_consensus.cc:2229] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 19:40:45.992910 31390 raft_consensus.cc:2258] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 19:40:45.995880 31389 ts_tablet_manager.cc:1918] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:40:45.996129 31389 log.cc:1196] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-2/wal/wals/7767d880bcbf487785e7161fd03e5cf3
I20260806 19:40:45.996582 31390 ts_tablet_manager.cc:1905] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:45.996905 31389 ts_tablet_manager.cc:1939] T 7767d880bcbf487785e7161fd03e5cf3 P 5045d9f2e90244f59f50fc28dfe63a0e: Deleting consensus metadata
I20260806 19:40:45.999197 30873 catalog_manager.cc:4841] TS 5045d9f2e90244f59f50fc28dfe63a0e (127.10.25.3:43881): tablet 7767d880bcbf487785e7161fd03e5cf3 (table test-table [id=c03eca4005e24b73bcca5dda47739389]) successfully deleted
I20260806 19:40:46.003262 31390 ts_tablet_manager.cc:1918] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 19:40:46.003507 31390 log.cc:1196] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5: Deleting WAL directory at /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786044861763400-10340-0/minicluster-data/ts-1/wal/wals/7767d880bcbf487785e7161fd03e5cf3
I20260806 19:40:46.004035 31390 ts_tablet_manager.cc:1939] T 7767d880bcbf487785e7161fd03e5cf3 P 1578720b4b9b40b29242cb634983bbe5: Deleting consensus metadata
I20260806 19:40:46.005905 30871 catalog_manager.cc:4841] TS 1578720b4b9b40b29242cb634983bbe5 (127.10.25.2:33961): tablet 7767d880bcbf487785e7161fd03e5cf3 (table test-table [id=c03eca4005e24b73bcca5dda47739389]) successfully deleted
I20260806 19:40:45.993309 31391 ts_tablet_manager.cc:1905] T 7767d880bcbf487785e7161fd03e5cf3 P 858c2dcf90bb46a4abbe541d20824d8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 19:40:46.110635 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 30948
I20260806 19:40:46.131870 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 31085
W20260806 19:40:46.131861 30873 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.25.1:37463 (error 108)
I20260806 19:40:46.134186 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:46.133664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:37463 (local address 127.0.0.1:55918)
0806 19:40:46.133874 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:46.133980 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:37463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 19:40:46.134517 30873 catalog_manager.cc:4584] TS 858c2dcf90bb46a4abbe541d20824d8f (127.10.25.1:37463): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7767d880bcbf487785e7161fd03e5cf3: Network error: Client connection negotiation failed: client connection to 127.10.25.1:37463: connect: Connection refused (error 111)
I20260806 19:40:46.140717 31364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:46.132935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52704 (local address 127.10.25.2:33961)
0806 19:40:46.133170 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:46.133176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:46.133202 (+ 26us) server_negotiation.cc:407] Connection header received
0806 19:40:46.133368 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:46.133375 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:46.133541 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:46.133654 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:46.134146 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:46.135215 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:46.136661 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:46.137102 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:46.137227 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:46.138775 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:46.138785 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:46.138800 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:46.139474 (+ 674us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:46.139529 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:46.139552 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:46.139669 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:46.140100 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 19:40:46.140236 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 19:40:46.141741 31210 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:46.188324 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:46.187746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:37463 (local address 127.0.0.1:55924)
0806 19:40:46.188023 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:46.188141 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:37463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260806 19:40:46.189850 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:46.189187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:37463 (local address 127.0.0.1:55926)
0806 19:40:46.189362 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:46.189451 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:37463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 19:40:46.238560 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 31085
I20260806 19:40:46.250983 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:46.250442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:37463 (local address 127.0.0.1:55936)
0806 19:40:46.250683 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:46.250771 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:37463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 19:40:46.252092 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:46.251724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:37463 (local address 127.0.0.1:55946)
0806 19:40:46.251873 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:46.251956 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:37463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 19:40:46.262503 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 31219
I20260806 19:40:46.271615 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:46.263647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49110 (local address 127.10.25.3:43881)
0806 19:40:46.263861 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:46.263868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:46.263893 (+ 25us) server_negotiation.cc:407] Connection header received
0806 19:40:46.264040 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:46.264046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:46.264152 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:46.264294 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:46.264772 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:46.265631 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:46.267028 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:46.267515 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:46.267679 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:46.269450 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:46.269465 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:46.269843 (+ 378us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:46.270566 (+ 723us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:46.270632 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:46.270665 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:46.270861 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:46.271070 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 19:40:46.271212 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 19:40:46.272379 31342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:46.333083 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:46.332520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:37463 (local address 127.0.0.1:55958)
0806 19:40:46.332785 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:46.332888 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:37463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260806 19:40:46.334333 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:46.333907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.1:37463 (local address 127.0.0.1:55960)
0806 19:40:46.334135 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 19:40:46.334219 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.1:37463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260806 19:40:46.371031 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 31219
I20260806 19:40:46.391780 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 30855
I20260806 19:40:46.393151 30918 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:46.515065 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 30855
2026-08-06T19:40:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3269 ms)
[----------] 6 tests from EncryptionPolicyTest (26113 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:40:46 dist-test-slave-ss4n krb5kdc[31400](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:40:46 dist-test-slave-ss4n krb5kdc[31400](info): set up 2 sockets
Aug 06 19:40:46 dist-test-slave-ss4n krb5kdc[31400](info): commencing operation
krb5kdc: starting...
W20260806 19:40:47.750175 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:40:47 dist-test-slave-ss4n krb5kdc[31400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:40:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:40:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:40:48.102793 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:48.480860 31413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:48.481314 31413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:48.481554 31413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:48.489878 31413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:40:48.490087 31413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:48.490202 31413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:48.490317 31413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:40:48.490409 31413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:40:48.502316 31413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38897
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:48.503901 31413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:48.505393 31413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:48.519665 31418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:48.519766 31419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:48.519747 31421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:48.520989 31413 server_base.cc:1034] running on GCE node
I20260806 19:40:48.522058 31413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:48.523983 31413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:48.525372 31413 hybrid_clock.cc:648] HybridClock initialized: now 1786045248525311 us; error 62 us; skew 500 ppm
I20260806 19:40:48.525885 31413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:48 dist-test-slave-ss4n krb5kdc[31400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045248, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:48.535894 31413 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:40:48.538192 31413 webserver.cc:460] Webserver started at http://127.0.0.1:35037/ using document root <none> and password file <none>
I20260806 19:40:48.538959 31413 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:48.539083 31413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:48.539366 31413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:48.543701 31413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "2cd33d480e384f5091cb60f3a16df4cb"
format_stamp: "Formatted at 2026-08-06 19:40:48 on dist-test-slave-ss4n"
server_key: "0bcfa527b2cd2c3ab78bc06f3a58e0f8"
server_key_iv: "7ca146a9de2a7c18bab81f097896a5a0"
server_key_version: "encryptionkey@0"
I20260806 19:40:48.544732 31413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "2cd33d480e384f5091cb60f3a16df4cb"
format_stamp: "Formatted at 2026-08-06 19:40:48 on dist-test-slave-ss4n"
server_key: "0bcfa527b2cd2c3ab78bc06f3a58e0f8"
server_key_iv: "7ca146a9de2a7c18bab81f097896a5a0"
server_key_version: "encryptionkey@0"
I20260806 19:40:48.551573 31413 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260806 19:40:48.556945 31428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:48.559415 31413 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 19:40:48.559738 31413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "2cd33d480e384f5091cb60f3a16df4cb"
format_stamp: "Formatted at 2026-08-06 19:40:48 on dist-test-slave-ss4n"
server_key: "0bcfa527b2cd2c3ab78bc06f3a58e0f8"
server_key_iv: "7ca146a9de2a7c18bab81f097896a5a0"
server_key_version: "encryptionkey@0"
I20260806 19:40:48.560019 31413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:48.585104 31413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:48.591310 31413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:48.591629 31413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:48.614929 31413 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38897
I20260806 19:40:48.614952 31480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38897 every 8 connection(s)
I20260806 19:40:48.616550 31413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:40:48.620379 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 31413
I20260806 19:40:48.620630 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:40:48.621505 10340 external_mini_cluster.cc:1428] Setting key 21e58f0d98e706109da1ea451072cad2
I20260806 19:40:48.622305 31481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:48.640183 31481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb: Bootstrap starting.
Aug 06 19:40:48 dist-test-slave-ss4n krb5kdc[31400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 19:40:48.644904 31481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:48.646601 31481 log.cc:824] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:48.651162 31481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb: No bootstrap required, opened a new log
I20260806 19:40:48.655112 31484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:48.623819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59401 (local address 127.0.0.1:38897)
0806 19:40:48.624784 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:48.624797 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:48.624844 (+ 47us) server_negotiation.cc:407] Connection header received
0806 19:40:48.627813 (+ 2969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:48.627845 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:48.628413 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:48.628862 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:48.631424 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:48.633250 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:48.635778 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:48.636861 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:48.637100 (+ 239us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:48.644230 (+ 7130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:48.644245 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:48.644268 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:48.644331 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:48.648985 (+ 4654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:48.650324 (+ 1339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:48.650334 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:48.650345 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:48.650598 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:48.651199 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:48.651204 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:48.651208 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:48.652053 (+ 845us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:48.652430 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:48.652977 (+ 547us) server_negotiation.cc:299] Negotiation successful
0806 19:40:48.653400 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":469,"threads_started":1}
I20260806 19:40:48.662475 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:40:48.662532 31481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd33d480e384f5091cb60f3a16df4cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } }
I20260806 19:40:48.663043 31481 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:48.663170 31481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd33d480e384f5091cb60f3a16df4cb, State: Initialized, Role: FOLLOWER
I20260806 19:40:48.663800 31481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd33d480e384f5091cb60f3a16df4cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } }
I20260806 19:40:48.664072 31481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:40:48.664240 31481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:40:48.664456 31481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:48.667399 31481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd33d480e384f5091cb60f3a16df4cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } }
I20260806 19:40:48.668061 31481 leader_election.cc:302] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cd33d480e384f5091cb60f3a16df4cb; no voters:
I20260806 19:40:48.668777 31481 leader_election.cc:288] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:40:48.668948 31490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:48.671245 31490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [term 1 LEADER]: Becoming Leader. State: Replica: 2cd33d480e384f5091cb60f3a16df4cb, State: Running, Role: LEADER
I20260806 19:40:48.671588 31481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:40:48.671924 31490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd33d480e384f5091cb60f3a16df4cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } }
I20260806 19:40:48.678078 31492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cd33d480e384f5091cb60f3a16df4cb. Latest consensus state: current_term: 1 leader_uuid: "2cd33d480e384f5091cb60f3a16df4cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd33d480e384f5091cb60f3a16df4cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } } }
I20260806 19:40:48.678511 31492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:48.677453 31491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cd33d480e384f5091cb60f3a16df4cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd33d480e384f5091cb60f3a16df4cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38897 } } }
I20260806 19:40:48.679363 31491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:48.680052 31501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:40:48.686991 31501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:40:48.698164 31484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:48.670415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56815 (local address 127.0.0.1:38897)
0806 19:40:48.670631 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:48.670638 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:48.670978 (+ 340us) server_negotiation.cc:407] Connection header received
0806 19:40:48.671415 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:48.671420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:48.671520 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:48.671616 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:48.674213 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:48.675934 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:48.680337 (+ 4403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:48.681407 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:48.681662 (+ 255us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:48.685577 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:48.685588 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:48.685595 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:48.685650 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:48.690702 (+ 5052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:48.692401 (+ 1699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:48.692410 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:48.692416 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:48.692660 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:48.693735 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:48.693744 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:48.693749 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:48.694358 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:48.694797 (+ 439us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:48.697451 (+ 2654us) server_negotiation.cc:299] Negotiation successful
0806 19:40:48.697738 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 19:40:48.700265 31501 catalog_manager.cc:1347] Generated new cluster ID: c63ec43a7dd9484c8aabc8658791e6a6
I20260806 19:40:48.700410 31501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:40:48.739867 31501 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:40:48.741621 31501 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:40:48.754882 31501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cd33d480e384f5091cb60f3a16df4cb: Generated new TSK 0
I20260806 19:40:48.755991 31501 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:40:48.770987 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 31413
I20260806 19:40:48.773936 31476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:48.879403 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 31413
W20260806 19:40:48.900898 31398 connection.cc:535] client connection to 127.0.0.1:38897 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38897): Connection reset by peer (error 104)
2026-08-06T19:40:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2353 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:40:49 dist-test-slave-ss4n krb5kdc[31513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:40:49 dist-test-slave-ss4n krb5kdc[31513](info): set up 2 sockets
Aug 06 19:40:49 dist-test-slave-ss4n krb5kdc[31513](info): commencing operation
krb5kdc: starting...
W20260806 19:40:50.127728 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:40:50 dist-test-slave-ss4n krb5kdc[31513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:40:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:40:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:40:50.482853 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39611
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:50.860850 31526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:50.861199 31526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:50.861382 31526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:50.869987 31526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:40:50.870138 31526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:50.870226 31526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:50.870318 31526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:40:50.870391 31526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:40:50.881233 31526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39611
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:50.882769 31526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:50.884191 31526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:50.898461 31531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:50.898401 31532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:50.898725 31534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:50.900697 31526 server_base.cc:1034] running on GCE node
I20260806 19:40:50.901793 31526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:50.903541 31526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:50.904867 31526 hybrid_clock.cc:648] HybridClock initialized: now 1786045250904825 us; error 43 us; skew 500 ppm
I20260806 19:40:50.905340 31526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:50 dist-test-slave-ss4n krb5kdc[31513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045250, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:50.909088 31526 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:40:50.911027 31526 webserver.cc:460] Webserver started at http://127.0.0.1:39767/ using document root <none> and password file <none>
I20260806 19:40:50.911808 31526 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:50.911936 31526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:50.912242 31526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:50.916359 31526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "40503842463342408cb9ebedfb6164b8"
format_stamp: "Formatted at 2026-08-06 19:40:50 on dist-test-slave-ss4n"
server_key: "6116160c05ba7ef699a2f15a4f16663b"
server_key_iv: "98170a0408b066d87206d4b59e75b12a"
server_key_version: "encryptionkey@0"
I20260806 19:40:50.917218 31526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "40503842463342408cb9ebedfb6164b8"
format_stamp: "Formatted at 2026-08-06 19:40:50 on dist-test-slave-ss4n"
server_key: "6116160c05ba7ef699a2f15a4f16663b"
server_key_iv: "98170a0408b066d87206d4b59e75b12a"
server_key_version: "encryptionkey@0"
I20260806 19:40:50.924466 31526 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 19:40:50.929435 31541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:50.931314 31526 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:40:50.931567 31526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "40503842463342408cb9ebedfb6164b8"
format_stamp: "Formatted at 2026-08-06 19:40:50 on dist-test-slave-ss4n"
server_key: "6116160c05ba7ef699a2f15a4f16663b"
server_key_iv: "98170a0408b066d87206d4b59e75b12a"
server_key_version: "encryptionkey@0"
I20260806 19:40:50.931834 31526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:50.977402 31526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:50.984359 31526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:50.984764 31526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:51.009598 31526 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39611
I20260806 19:40:51.009471 31592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39611 every 8 connection(s)
I20260806 19:40:51.011292 31526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:40:51.016824 31593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:51.021237 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 31526
I20260806 19:40:51.021473 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:40:51.022142 10340 external_mini_cluster.cc:1428] Setting key 4b3c3c262f9054dcb388db70653c4c11
I20260806 19:40:51.031016 31593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8: Bootstrap starting.
I20260806 19:40:51.034665 31593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:51.035897 31593 log.cc:824] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:51.037755 31596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:51.024359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45151 (local address 127.0.0.1:39611)
0806 19:40:51.025184 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:51.025197 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:51.025242 (+ 45us) server_negotiation.cc:407] Connection header received
0806 19:40:51.027159 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:51.027184 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:51.027648 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:51.028109 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:51.028761 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:51.030242 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:51.031725 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:51.032441 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:51.032689 (+ 248us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:51.034282 (+ 1593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:51.034291 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:51.034309 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:51.035019 (+ 710us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:51.035124 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:51.035165 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:51.035354 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:51.035822 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 19:40:51.036071 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":342,"threads_started":1}
I20260806 19:40:51.039749 31593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8: No bootstrap required, opened a new log
I20260806 19:40:51.043170 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:40:51.050897 31593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40503842463342408cb9ebedfb6164b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39611 } }
I20260806 19:40:51.051460 31593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:51.051608 31593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40503842463342408cb9ebedfb6164b8, State: Initialized, Role: FOLLOWER
I20260806 19:40:51.052368 31593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40503842463342408cb9ebedfb6164b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39611 } }
I20260806 19:40:51.052676 31593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:40:51.052855 31593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:40:51.053056 31593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:51.057262 31593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40503842463342408cb9ebedfb6164b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39611 } }
I20260806 19:40:51.057941 31596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:51.049311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34469 (local address 127.0.0.1:39611)
0806 19:40:51.049574 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:51.049584 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:51.049992 (+ 408us) server_negotiation.cc:407] Connection header received
0806 19:40:51.050254 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:51.050259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:51.050332 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:51.050466 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:51.051124 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:51.052179 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:51.054057 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:51.054813 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:51.054984 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:51.056784 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:51.056789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:51.056793 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:51.056815 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:51.056884 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:51.056902 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:51.057051 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:51.057275 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 19:40:51.057342 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 19:40:51.058166 31593 leader_election.cc:302] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40503842463342408cb9ebedfb6164b8; no voters:
I20260806 19:40:51.059026 31593 leader_election.cc:288] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:40:51.059181 31603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:51.062342 31593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:40:51.063794 31603 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [term 1 LEADER]: Becoming Leader. State: Replica: 40503842463342408cb9ebedfb6164b8, State: Running, Role: LEADER
I20260806 19:40:51.064723 31603 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40503842463342408cb9ebedfb6164b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39611 } }
I20260806 19:40:51.068823 31605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40503842463342408cb9ebedfb6164b8. Latest consensus state: current_term: 1 leader_uuid: "40503842463342408cb9ebedfb6164b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40503842463342408cb9ebedfb6164b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39611 } } }
I20260806 19:40:51.068938 31604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40503842463342408cb9ebedfb6164b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40503842463342408cb9ebedfb6164b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39611 } } }
I20260806 19:40:51.069334 31605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:51.069429 31604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8 [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:51.074308 31611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:40:51.080577 31611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:40:51.091631 31611 catalog_manager.cc:1347] Generated new cluster ID: 84c15facdf4949a8b3705c065a265a90
I20260806 19:40:51.091763 31611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:40:51.103135 31611 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:40:51.104583 31611 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:40:51.112545 31611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40503842463342408cb9ebedfb6164b8: Generated new TSK 0
I20260806 19:40:51.113603 31611 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:40:51.123646 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 31526
W20260806 19:40:51.125092 31588 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45151
W20260806 19:40:51.126032 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 31526: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:40:51.126252 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 31526
W20260806 19:40:51.145205 31511 connection.cc:535] client connection to 127.0.0.1:39611 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39611): Connection reset by peer (error 104)
2026-08-06T19:40:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2243 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:40:51 dist-test-slave-ss4n krb5kdc[31624](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:40:51 dist-test-slave-ss4n krb5kdc[31624](info): set up 2 sockets
Aug 06 19:40:51 dist-test-slave-ss4n krb5kdc[31624](info): commencing operation
krb5kdc: starting...
W20260806 19:40:52.401382 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:40:52 dist-test-slave-ss4n krb5kdc[31624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:40:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:40:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:40:52.757979 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34567
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:53.133252 31637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:53.133674 31637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:53.133780 31637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:53.141430 31637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:40:53.141616 31637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:53.141669 31637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:53.141726 31637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:40:53.141773 31637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:40:53.152107 31637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34567
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:53.153615 31637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:53.155017 31637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:53.167514 31645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:53.167585 31643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:53.167611 31642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:53.168361 31637 server_base.cc:1034] running on GCE node
I20260806 19:40:53.169325 31637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:53.170926 31637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:53.172201 31637 hybrid_clock.cc:648] HybridClock initialized: now 1786045253172156 us; error 66 us; skew 500 ppm
I20260806 19:40:53.172600 31637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:53 dist-test-slave-ss4n krb5kdc[31624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045253, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:53.176113 31637 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:40:53.178149 31637 webserver.cc:460] Webserver started at http://127.0.0.1:38873/ using document root <none> and password file <none>
I20260806 19:40:53.178972 31637 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:53.179153 31637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:53.179522 31637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:53.183969 31637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "46a10f568b6b425f937b8a87614f2edc"
format_stamp: "Formatted at 2026-08-06 19:40:53 on dist-test-slave-ss4n"
server_key: "c8cb86309398b1daf68b1eaab0ed455f"
server_key_iv: "6f911b1ff0a4700696e6b031828cbeb7"
server_key_version: "encryptionkey@0"
I20260806 19:40:53.185074 31637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "46a10f568b6b425f937b8a87614f2edc"
format_stamp: "Formatted at 2026-08-06 19:40:53 on dist-test-slave-ss4n"
server_key: "c8cb86309398b1daf68b1eaab0ed455f"
server_key_iv: "6f911b1ff0a4700696e6b031828cbeb7"
server_key_version: "encryptionkey@0"
I20260806 19:40:53.192514 31637 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 19:40:53.196892 31652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:53.198447 31637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 19:40:53.198697 31637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "46a10f568b6b425f937b8a87614f2edc"
format_stamp: "Formatted at 2026-08-06 19:40:53 on dist-test-slave-ss4n"
server_key: "c8cb86309398b1daf68b1eaab0ed455f"
server_key_iv: "6f911b1ff0a4700696e6b031828cbeb7"
server_key_version: "encryptionkey@0"
I20260806 19:40:53.198932 31637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:53.205255 31637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:53.209951 31637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:53.210282 31637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:53.232134 31637 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34567
I20260806 19:40:53.232188 31706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34567 every 8 connection(s)
I20260806 19:40:53.233779 31637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:40:53.234422 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 31637
I20260806 19:40:53.234632 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:40:53.235261 10340 external_mini_cluster.cc:1428] Setting key e2e1ac1ab9b29bf0dca134809ac76f75
I20260806 19:40:53.238417 31707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:53.243443 31710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:53.237250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44979 (local address 127.0.0.1:34567)
0806 19:40:53.238128 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:53.238140 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:53.238190 (+ 50us) server_negotiation.cc:407] Connection header received
0806 19:40:53.239855 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:53.239877 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:53.240232 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:53.240561 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:53.241757 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:53.241763 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:53.241777 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:53.241793 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:53.241875 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:53.241896 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:53.242080 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:53.242210 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 19:40:53.242441 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":420,"threads_started":1}
I20260806 19:40:53.247968 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:40:53.251235 31707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc: Bootstrap starting.
I20260806 19:40:53.255097 31707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:53.256492 31707 log.cc:824] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:53.256613 31710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:53.253538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36657 (local address 127.0.0.1:34567)
0806 19:40:53.253819 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:53.253828 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:53.254226 (+ 398us) server_negotiation.cc:407] Connection header received
0806 19:40:53.254438 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:53.254442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:53.254496 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:53.254590 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:53.255833 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:53.255838 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:53.255842 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:53.255862 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:53.255911 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:53.255934 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:53.256060 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:53.256198 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 19:40:53.256276 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 19:40:53.259792 31707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc: No bootstrap required, opened a new log
I20260806 19:40:53.267771 31707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a10f568b6b425f937b8a87614f2edc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34567 } }
I20260806 19:40:53.268242 31707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:53.268322 31707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a10f568b6b425f937b8a87614f2edc, State: Initialized, Role: FOLLOWER
I20260806 19:40:53.268853 31707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a10f568b6b425f937b8a87614f2edc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34567 } }
I20260806 19:40:53.269058 31707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:40:53.269176 31707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:40:53.269301 31707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:53.271600 31707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a10f568b6b425f937b8a87614f2edc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34567 } }
I20260806 19:40:53.272109 31707 leader_election.cc:302] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46a10f568b6b425f937b8a87614f2edc; no voters:
I20260806 19:40:53.272689 31707 leader_election.cc:288] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:40:53.272902 31717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:53.273754 31717 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [term 1 LEADER]: Becoming Leader. State: Replica: 46a10f568b6b425f937b8a87614f2edc, State: Running, Role: LEADER
I20260806 19:40:53.274434 31717 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a10f568b6b425f937b8a87614f2edc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34567 } }
I20260806 19:40:53.275115 31707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:40:53.278074 31720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46a10f568b6b425f937b8a87614f2edc. Latest consensus state: current_term: 1 leader_uuid: "46a10f568b6b425f937b8a87614f2edc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a10f568b6b425f937b8a87614f2edc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34567 } } }
I20260806 19:40:53.278621 31720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:53.278107 31718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46a10f568b6b425f937b8a87614f2edc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a10f568b6b425f937b8a87614f2edc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34567 } } }
I20260806 19:40:53.279393 31718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:53.283268 31726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:40:53.289350 31726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:40:53.302052 31726 catalog_manager.cc:1347] Generated new cluster ID: 2ad0369d6b0d47f4a72e6108cd22f7e0
I20260806 19:40:53.302248 31726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:40:53.316483 31726 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:40:53.317028 31726 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:40:53.324735 31726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46a10f568b6b425f937b8a87614f2edc: Generated new TSK 0
I20260806 19:40:53.326248 31726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:40:53.343189 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 31637
W20260806 19:40:53.344563 31701 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44979
W20260806 19:40:53.345417 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 31637: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:40:53.345646 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 31637
2026-08-06T19:40:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2221 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:40:53 dist-test-slave-ss4n krb5kdc[31740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:40:53 dist-test-slave-ss4n krb5kdc[31740](info): set up 2 sockets
Aug 06 19:40:53 dist-test-slave-ss4n krb5kdc[31740](info): commencing operation
krb5kdc: starting...
W20260806 19:40:55.011009 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.624s user 0.011s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:40:55 dist-test-slave-ss4n krb5kdc[31740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:40:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:40:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:40:55.370584 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34117
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:55.741142 31753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:55.741556 31753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:55.741721 31753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:55.751389 31753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:40:55.751564 31753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:55.751617 31753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:55.751679 31753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:40:55.751718 31753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:40:55.763999 31753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:55.766202 31753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:55.768245 31753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:55.784884 31761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:55.785122 31759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:55.785162 31758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:55.786149 31753 server_base.cc:1034] running on GCE node
I20260806 19:40:55.787119 31753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:55.788834 31753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:55.790098 31753 hybrid_clock.cc:648] HybridClock initialized: now 1786045255790038 us; error 54 us; skew 500 ppm
I20260806 19:40:55.790550 31753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:55 dist-test-slave-ss4n krb5kdc[31740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045255, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:55.794868 31753 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:40:55.796849 31753 webserver.cc:460] Webserver started at http://127.0.0.1:33091/ using document root <none> and password file <none>
I20260806 19:40:55.798125 31753 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:55.798323 31753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:55.798707 31753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:55.802539 31753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "02bab0304d674d7b9295c361c76df65c"
format_stamp: "Formatted at 2026-08-06 19:40:55 on dist-test-slave-ss4n"
server_key: "67421d6bad926e47150addb5f2bd3608"
server_key_iv: "6bd44ee5c08d187f0c68e1f3cf6eab0b"
server_key_version: "encryptionkey@0"
I20260806 19:40:55.803612 31753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "02bab0304d674d7b9295c361c76df65c"
format_stamp: "Formatted at 2026-08-06 19:40:55 on dist-test-slave-ss4n"
server_key: "67421d6bad926e47150addb5f2bd3608"
server_key_iv: "6bd44ee5c08d187f0c68e1f3cf6eab0b"
server_key_version: "encryptionkey@0"
I20260806 19:40:55.810568 31753 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260806 19:40:55.815448 31768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:55.817281 31753 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 19:40:55.817618 31753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "02bab0304d674d7b9295c361c76df65c"
format_stamp: "Formatted at 2026-08-06 19:40:55 on dist-test-slave-ss4n"
server_key: "67421d6bad926e47150addb5f2bd3608"
server_key_iv: "6bd44ee5c08d187f0c68e1f3cf6eab0b"
server_key_version: "encryptionkey@0"
I20260806 19:40:55.817893 31753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:55.842577 31753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:55.848004 31753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:55.848393 31753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:55.874619 31753 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34117
I20260806 19:40:55.874676 31821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34117 every 8 connection(s)
I20260806 19:40:55.876276 31753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:40:55.878279 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 31753
I20260806 19:40:55.878532 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:40:55.879459 10340 external_mini_cluster.cc:1428] Setting key 4d68374187b8446d3f20f79fd8971c22
I20260806 19:40:55.881152 31822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:55.898963 31822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c: Bootstrap starting.
Aug 06 19:40:55 dist-test-slave-ss4n krb5kdc[31740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 19:40:55.902908 31822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:55.904539 31822 log.cc:824] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:55.908490 31822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c: No bootstrap required, opened a new log
I20260806 19:40:55.912061 31825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:55.882317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56627 (local address 127.0.0.1:34117)
0806 19:40:55.883308 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:55.883321 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:55.883369 (+ 48us) server_negotiation.cc:407] Connection header received
0806 19:40:55.886311 (+ 2942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:55.886343 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:55.886953 (+ 610us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:55.887425 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:55.889969 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:55.892646 (+ 2677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:55.895065 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:55.896080 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:55.896258 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:55.902037 (+ 5779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:55.902047 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:55.902064 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:55.902118 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:55.906333 (+ 4215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:55.907494 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:55.907504 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:55.907515 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:55.907758 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:55.908246 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:55.908251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:55.908254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:55.909056 (+ 802us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:55.909435 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:55.909946 (+ 511us) server_negotiation.cc:299] Negotiation successful
0806 19:40:55.910468 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":759,"thread_start_us":490,"threads_started":1}
I20260806 19:40:55.918295 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:40:55.920913 31822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02bab0304d674d7b9295c361c76df65c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34117 } }
I20260806 19:40:55.921555 31822 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:55.921787 31822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02bab0304d674d7b9295c361c76df65c, State: Initialized, Role: FOLLOWER
I20260806 19:40:55.922760 31822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02bab0304d674d7b9295c361c76df65c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34117 } }
I20260806 19:40:55.923259 31822 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:40:55.923454 31822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:40:55.923679 31822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:55.926743 31822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02bab0304d674d7b9295c361c76df65c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34117 } }
I20260806 19:40:55.927371 31822 leader_election.cc:302] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02bab0304d674d7b9295c361c76df65c; no voters:
I20260806 19:40:55.928218 31822 leader_election.cc:288] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:40:55.928419 31832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:55.931149 31822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:40:55.936708 31832 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [term 1 LEADER]: Becoming Leader. State: Replica: 02bab0304d674d7b9295c361c76df65c, State: Running, Role: LEADER
I20260806 19:40:55.937450 31832 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02bab0304d674d7b9295c361c76df65c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34117 } }
I20260806 19:40:55.942764 31834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02bab0304d674d7b9295c361c76df65c. Latest consensus state: current_term: 1 leader_uuid: "02bab0304d674d7b9295c361c76df65c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02bab0304d674d7b9295c361c76df65c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34117 } } }
I20260806 19:40:55.943252 31834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:55.943866 31833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02bab0304d674d7b9295c361c76df65c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02bab0304d674d7b9295c361c76df65c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34117 } } }
I20260806 19:40:55.944164 31833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:55.946346 31846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:40:55.946688 31825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:55.924549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43211 (local address 127.0.0.1:34117)
0806 19:40:55.924759 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:55.924765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:55.925293 (+ 528us) server_negotiation.cc:407] Connection header received
0806 19:40:55.925444 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:55.925450 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:55.925563 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:55.925696 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:55.927513 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:55.928750 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:55.930854 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:55.931563 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:55.931685 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:55.936593 (+ 4908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:55.936604 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:55.936613 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:40:55.936684 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:55.942218 (+ 5534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:55.943519 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:55.943527 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:55.943533 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:55.943741 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:40:55.944334 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:40:55.944339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:40:55.944342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:40:55.945014 (+ 672us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:55.945365 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:55.945966 (+ 601us) server_negotiation.cc:299] Negotiation successful
0806 19:40:55.946228 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 19:40:55.955056 31846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:40:55.966065 31846 catalog_manager.cc:1347] Generated new cluster ID: e737eb96fad845c7a260a13b7215ef22
I20260806 19:40:55.966231 31846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:40:56.001590 31846 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:40:56.003294 31846 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:40:56.012300 31846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02bab0304d674d7b9295c361c76df65c: Generated new TSK 0
I20260806 19:40:56.013562 31846 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:40:56.027000 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 31753
I20260806 19:40:56.028367 31817 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:40:56.131639 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 31753
2026-08-06T19:40:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2784 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:40:56 dist-test-slave-ss4n krb5kdc[31855](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:40:56 dist-test-slave-ss4n krb5kdc[31855](info): set up 2 sockets
Aug 06 19:40:56 dist-test-slave-ss4n krb5kdc[31855](info): commencing operation
krb5kdc: starting...
W20260806 19:40:57.354017 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.006s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:40:57 dist-test-slave-ss4n krb5kdc[31855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:40:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:40:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:40:57.715708 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42019
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:40:58.113605 31868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:40:58.113996 31868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:40:58.114163 31868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:40:58.122711 31868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:40:58.122895 31868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:40:58.122956 31868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:40:58.123107 31868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:40:58.123188 31868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:40:58.136052 31868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:40:58.137655 31868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:40:58.139282 31868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:40:58.153362 31878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:58.153586 31875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:40:58.153565 31876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:40:58.154264 31868 server_base.cc:1034] running on GCE node
I20260806 19:40:58.155172 31868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:40:58.156847 31868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:40:58.158156 31868 hybrid_clock.cc:648] HybridClock initialized: now 1786045258158105 us; error 62 us; skew 500 ppm
I20260806 19:40:58.158667 31868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:40:58 dist-test-slave-ss4n krb5kdc[31855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045258, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:40:58.162480 31868 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:40:58.164487 31868 webserver.cc:460] Webserver started at http://127.0.0.1:40397/ using document root <none> and password file <none>
I20260806 19:40:58.165966 31868 fs_manager.cc:359] Metadata directory not provided
I20260806 19:40:58.166224 31868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:40:58.166682 31868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:40:58.170425 31868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "81b2488749ea42bcaeec7a5a4d642750"
format_stamp: "Formatted at 2026-08-06 19:40:58 on dist-test-slave-ss4n"
server_key: "719ef0322b148169d2eee8d47143ece8"
server_key_iv: "66f5a367b083d69401dab72eece49782"
server_key_version: "encryptionkey@0"
I20260806 19:40:58.171427 31868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "81b2488749ea42bcaeec7a5a4d642750"
format_stamp: "Formatted at 2026-08-06 19:40:58 on dist-test-slave-ss4n"
server_key: "719ef0322b148169d2eee8d47143ece8"
server_key_iv: "66f5a367b083d69401dab72eece49782"
server_key_version: "encryptionkey@0"
I20260806 19:40:58.178172 31868 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 19:40:58.182881 31886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:40:58.184947 31868 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 19:40:58.185214 31868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "81b2488749ea42bcaeec7a5a4d642750"
format_stamp: "Formatted at 2026-08-06 19:40:58 on dist-test-slave-ss4n"
server_key: "719ef0322b148169d2eee8d47143ece8"
server_key_iv: "66f5a367b083d69401dab72eece49782"
server_key_version: "encryptionkey@0"
I20260806 19:40:58.185431 31868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:40:58.202486 31868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:40:58.209400 31868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:40:58.209853 31868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:40:58.233915 31868 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42019
I20260806 19:40:58.234143 31938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42019 every 8 connection(s)
I20260806 19:40:58.235661 31868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:40:58.240159 31939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:40:58.243330 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 31868
I20260806 19:40:58.243566 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:40:58.244282 10340 external_mini_cluster.cc:1428] Setting key 5bb4da18013eab43f8c4c2fe5b69c6c2
I20260806 19:40:58.256040 31939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750: Bootstrap starting.
I20260806 19:40:58.260628 31939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750: Neither blocks nor log segments found. Creating new log.
I20260806 19:40:58.262055 31939 log.cc:824] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750: Log is configured to *not* fsync() on all Append() calls
I20260806 19:40:58.262475 31942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:58.246574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40685 (local address 127.0.0.1:42019)
0806 19:40:58.247634 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:58.247656 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:58.247720 (+ 64us) server_negotiation.cc:407] Connection header received
0806 19:40:58.249826 (+ 2106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:58.249855 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:58.250345 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:58.250750 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:58.251559 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:58.253235 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:58.255429 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:58.256757 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:58.256991 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:58.258838 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:58.258848 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:58.258876 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:58.259492 (+ 616us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:58.259594 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:58.259626 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:58.259748 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:58.260272 (+ 524us) server_negotiation.cc:299] Negotiation successful
0806 19:40:58.260748 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":445,"threads_started":1}
I20260806 19:40:58.265937 31939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750: No bootstrap required, opened a new log
I20260806 19:40:58.268035 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:40:58.278065 31939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b2488749ea42bcaeec7a5a4d642750" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42019 } }
I20260806 19:40:58.278674 31939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:40:58.278859 31939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81b2488749ea42bcaeec7a5a4d642750, State: Initialized, Role: FOLLOWER
I20260806 19:40:58.279708 31939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b2488749ea42bcaeec7a5a4d642750" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42019 } }
I20260806 19:40:58.280067 31939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:40:58.280257 31939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:40:58.280489 31939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:40:58.283174 31942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:40:58.274575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45287 (local address 127.0.0.1:42019)
0806 19:40:58.274762 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 19:40:58.274766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:40:58.275116 (+ 350us) server_negotiation.cc:407] Connection header received
0806 19:40:58.275385 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:40:58.275390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:40:58.275485 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:40:58.275594 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:40:58.276390 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:58.277262 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:58.279554 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:40:58.280553 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:40:58.280730 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:40:58.282278 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:40:58.282286 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:40:58.282290 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:40:58.282311 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:40:58.282358 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:40:58.282373 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:40:58.282458 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 19:40:58.282667 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 19:40:58.282799 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 19:40:58.284192 31939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b2488749ea42bcaeec7a5a4d642750" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42019 } }
I20260806 19:40:58.284816 31939 leader_election.cc:302] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81b2488749ea42bcaeec7a5a4d642750; no voters:
I20260806 19:40:58.285598 31939 leader_election.cc:288] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:40:58.285755 31949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:40:58.286537 31949 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [term 1 LEADER]: Becoming Leader. State: Replica: 81b2488749ea42bcaeec7a5a4d642750, State: Running, Role: LEADER
I20260806 19:40:58.287225 31949 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b2488749ea42bcaeec7a5a4d642750" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42019 } }
I20260806 19:40:58.288388 31939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:40:58.290288 31950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81b2488749ea42bcaeec7a5a4d642750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b2488749ea42bcaeec7a5a4d642750" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42019 } } }
I20260806 19:40:58.290778 31950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:58.290357 31951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81b2488749ea42bcaeec7a5a4d642750. Latest consensus state: current_term: 1 leader_uuid: "81b2488749ea42bcaeec7a5a4d642750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b2488749ea42bcaeec7a5a4d642750" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42019 } } }
I20260806 19:40:58.291219 31951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750 [sys.catalog]: This master's current role is: LEADER
I20260806 19:40:58.291702 31954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:40:58.299104 31954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:40:58.312738 31954 catalog_manager.cc:1347] Generated new cluster ID: 8b07e84530c542629dfe25dc0ae6f759
I20260806 19:40:58.312894 31954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:40:58.327466 31954 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:40:58.329226 31954 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:40:58.339655 31954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81b2488749ea42bcaeec7a5a4d642750: Generated new TSK 0
I20260806 19:40:58.340879 31954 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:40:58.360570 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 31868
W20260806 19:40:58.362459 31933 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40685
W20260806 19:40:58.363698 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 31868: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:40:58.363929 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 31868
2026-08-06T19:40:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2233 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:40:58 dist-test-slave-ss4n krb5kdc[31971](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:40:58 dist-test-slave-ss4n krb5kdc[31971](info): set up 2 sockets
Aug 06 19:40:58 dist-test-slave-ss4n krb5kdc[31971](info): commencing operation
krb5kdc: starting...
W20260806 19:40:59.642221 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:40:59 dist-test-slave-ss4n krb5kdc[31971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:40:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:40:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:41:00.018384 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42105
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:00.425932 31984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:00.426288 31984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:00.426390 31984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:00.434449 31984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:00.434599 31984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:00.434648 31984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:00.434702 31984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:00.434744 31984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:00.447443 31984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:00.449301 31984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:00.451781 31984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:00.471189 31989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:00.471195 31990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:00.471235 31992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:00.472914 31984 server_base.cc:1034] running on GCE node
I20260806 19:41:00.474138 31984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:00.478574 31984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:00.480070 31984 hybrid_clock.cc:648] HybridClock initialized: now 1786045260480007 us; error 91 us; skew 500 ppm
I20260806 19:41:00.481040 31984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:00 dist-test-slave-ss4n krb5kdc[31971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045260, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:00.490485 31984 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:41:00.493858 31984 webserver.cc:460] Webserver started at http://127.0.0.1:44943/ using document root <none> and password file <none>
I20260806 19:41:00.495334 31984 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:00.495805 31984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:00.496349 31984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:00.501169 31984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "e4b797f66cf046d89d7f4cde59195d68"
format_stamp: "Formatted at 2026-08-06 19:41:00 on dist-test-slave-ss4n"
server_key: "7c3dca6a4453aa0b0c7c64ec7336631f"
server_key_iv: "72bd1da6a27238d835e07ccad1665c93"
server_key_version: "encryptionkey@0"
I20260806 19:41:00.502332 31984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "e4b797f66cf046d89d7f4cde59195d68"
format_stamp: "Formatted at 2026-08-06 19:41:00 on dist-test-slave-ss4n"
server_key: "7c3dca6a4453aa0b0c7c64ec7336631f"
server_key_iv: "72bd1da6a27238d835e07ccad1665c93"
server_key_version: "encryptionkey@0"
I20260806 19:41:00.509759 31984 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 19:41:00.515277 32000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:00.518056 31984 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:41:00.518356 31984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "e4b797f66cf046d89d7f4cde59195d68"
format_stamp: "Formatted at 2026-08-06 19:41:00 on dist-test-slave-ss4n"
server_key: "7c3dca6a4453aa0b0c7c64ec7336631f"
server_key_iv: "72bd1da6a27238d835e07ccad1665c93"
server_key_version: "encryptionkey@0"
I20260806 19:41:00.518662 31984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:00.528232 31984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:00.534085 31984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:00.534502 31984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:00.563742 31984 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42105
I20260806 19:41:00.563795 32052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42105 every 8 connection(s)
I20260806 19:41:00.565379 31984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:00.569392 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 31984
I20260806 19:41:00.569646 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:41:00.569922 32053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:00.570245 10340 external_mini_cluster.cc:1428] Setting key 5617e0406e79802126564ec6591c4935
I20260806 19:41:00.579811 32056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:00.572229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54005 (local address 127.0.0.1:42105)
0806 19:41:00.573175 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:00.573191 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:00.573239 (+ 48us) server_negotiation.cc:407] Connection header received
0806 19:41:00.575140 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:00.575171 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:00.575565 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:00.576000 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:00.577288 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:00.577295 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:00.577312 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:00.577339 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:00.577409 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:00.577438 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:00.577848 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:00.577986 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 19:41:00.578261 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":431,"threads_started":1}
I20260806 19:41:00.585402 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:41:00.586105 32053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68: Bootstrap starting.
I20260806 19:41:00.591449 32053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:00.593034 32053 log.cc:824] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:00.596926 32056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:00.592384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50415 (local address 127.0.0.1:42105)
0806 19:41:00.592660 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:00.592667 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:00.593321 (+ 654us) server_negotiation.cc:407] Connection header received
0806 19:41:00.593637 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:00.593646 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:00.593913 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:00.594149 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:00.595820 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:00.595828 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:00.595834 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:00.595867 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:00.595943 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:00.595977 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:00.596165 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:00.596386 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 19:41:00.596477 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 19:41:00.598491 32053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68: No bootstrap required, opened a new log
I20260806 19:41:00.607358 32053 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b797f66cf046d89d7f4cde59195d68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42105 } }
I20260806 19:41:00.607933 32053 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:00.608084 32053 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4b797f66cf046d89d7f4cde59195d68, State: Initialized, Role: FOLLOWER
I20260806 19:41:00.608814 32053 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b797f66cf046d89d7f4cde59195d68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42105 } }
I20260806 19:41:00.609115 32053 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:41:00.609274 32053 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:41:00.609529 32053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:00.612087 32053 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b797f66cf046d89d7f4cde59195d68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42105 } }
I20260806 19:41:00.612697 32053 leader_election.cc:302] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4b797f66cf046d89d7f4cde59195d68; no voters:
I20260806 19:41:00.613399 32053 leader_election.cc:288] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:41:00.613588 32063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:00.614872 32063 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [term 1 LEADER]: Becoming Leader. State: Replica: e4b797f66cf046d89d7f4cde59195d68, State: Running, Role: LEADER
I20260806 19:41:00.615509 32063 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b797f66cf046d89d7f4cde59195d68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42105 } }
I20260806 19:41:00.615917 32053 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:00.620860 32065 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4b797f66cf046d89d7f4cde59195d68. Latest consensus state: current_term: 1 leader_uuid: "e4b797f66cf046d89d7f4cde59195d68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b797f66cf046d89d7f4cde59195d68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42105 } } }
I20260806 19:41:00.621366 32065 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:00.621807 32064 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4b797f66cf046d89d7f4cde59195d68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4b797f66cf046d89d7f4cde59195d68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42105 } } }
I20260806 19:41:00.622234 32064 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:00.623147 32073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:41:00.628265 32073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:00.638761 32073 catalog_manager.cc:1347] Generated new cluster ID: e93ccdad4db24ea0bd5d0b7e31214444
I20260806 19:41:00.638912 32073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:00.658278 32073 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:00.659262 32073 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:00.671873 32073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4b797f66cf046d89d7f4cde59195d68: Generated new TSK 0
I20260806 19:41:00.673139 32073 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:41:00.692961 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 31984
W20260806 19:41:00.695271 32048 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54005
W20260806 19:41:00.696350 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 31984: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:41:00.696590 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 31984
2026-08-06T19:41:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2335 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:00 dist-test-slave-ss4n krb5kdc[32084](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:00 dist-test-slave-ss4n krb5kdc[32084](info): set up 2 sockets
Aug 06 19:41:00 dist-test-slave-ss4n krb5kdc[32084](info): commencing operation
krb5kdc: starting...
W20260806 19:41:01.978781 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.004s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:41:02 dist-test-slave-ss4n krb5kdc[32084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:41:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:41:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:41:02.350793 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:02.767931 32098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:02.768344 32098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:02.768528 32098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:02.776379 32098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:02.776614 32098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:02.776736 32098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:02.776857 32098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:02.776942 32098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:02.787441 32098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:02.788997 32098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:02.790485 32098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:02.805308 32103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:02.805344 32106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:02.805373 32104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:02.807082 32098 server_base.cc:1034] running on GCE node
I20260806 19:41:02.808086 32098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:02.810518 32098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:02.811834 32098 hybrid_clock.cc:648] HybridClock initialized: now 1786045262811761 us; error 67 us; skew 500 ppm
I20260806 19:41:02.812280 32098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:02 dist-test-slave-ss4n krb5kdc[32084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045262, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:02.816184 32098 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:41:02.818140 32098 webserver.cc:460] Webserver started at http://127.0.0.1:37573/ using document root <none> and password file <none>
I20260806 19:41:02.818876 32098 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:02.819015 32098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:02.819453 32098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:02.823168 32098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "c05fe0af732c49c98184c3bfb1a1015a"
format_stamp: "Formatted at 2026-08-06 19:41:02 on dist-test-slave-ss4n"
server_key: "7314b200e35719b0129bb9607835d667"
server_key_iv: "84d015a756275e62cfbee85b0b889e9e"
server_key_version: "encryptionkey@0"
I20260806 19:41:02.824031 32098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "c05fe0af732c49c98184c3bfb1a1015a"
format_stamp: "Formatted at 2026-08-06 19:41:02 on dist-test-slave-ss4n"
server_key: "7314b200e35719b0129bb9607835d667"
server_key_iv: "84d015a756275e62cfbee85b0b889e9e"
server_key_version: "encryptionkey@0"
I20260806 19:41:02.830276 32098 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 19:41:02.835184 32113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:02.837446 32098 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:41:02.837815 32098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "c05fe0af732c49c98184c3bfb1a1015a"
format_stamp: "Formatted at 2026-08-06 19:41:02 on dist-test-slave-ss4n"
server_key: "7314b200e35719b0129bb9607835d667"
server_key_iv: "84d015a756275e62cfbee85b0b889e9e"
server_key_version: "encryptionkey@0"
I20260806 19:41:02.838104 32098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:02.855672 32098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:02.862021 32098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:02.862439 32098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:02.885664 32098 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42695
I20260806 19:41:02.885730 32164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42695 every 8 connection(s)
I20260806 19:41:02.887279 32098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:02.891165 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 32098
I20260806 19:41:02.891408 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:41:02.891937 10340 external_mini_cluster.cc:1428] Setting key 593e982ac97d339a38b1934a521ffc4d
I20260806 19:41:02.892145 32165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:02.909436 32165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a: Bootstrap starting.
Aug 06 19:41:02 dist-test-slave-ss4n krb5kdc[32084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 19:41:02.913224 32165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:02.915108 32165 log.cc:824] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:02.922041 32165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a: No bootstrap required, opened a new log
I20260806 19:41:02.928156 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:02.894352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:55151 (local address 127.0.0.1:42695)
0806 19:41:02.895777 (+ 1425us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:02.895800 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:02.895874 (+ 74us) server_negotiation.cc:407] Connection header received
0806 19:41:02.898671 (+ 2797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:02.898707 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:02.899592 (+ 885us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:02.900519 (+ 927us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:02.902401 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:02.903808 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:02.905814 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:02.906963 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:02.907278 (+ 315us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:02.913530 (+ 6252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:02.913547 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:02.913572 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:02.913647 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:02.919337 (+ 5690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:02.921513 (+ 2176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:02.921528 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:02.921539 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:02.921926 (+ 387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:02.922882 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:02.922888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:02.922892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:02.924328 (+ 1436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:02.924881 (+ 553us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:02.925911 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0806 19:41:02.926479 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1074,"thread_start_us":713,"threads_started":1}
I20260806 19:41:02.935587 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:41:02.935659 32165 raft_consensus.cc:348] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05fe0af732c49c98184c3bfb1a1015a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } }
I20260806 19:41:02.936214 32165 raft_consensus.cc:374] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:02.936329 32165 raft_consensus.cc:729] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c05fe0af732c49c98184c3bfb1a1015a, State: Initialized, Role: FOLLOWER
I20260806 19:41:02.937171 32165 consensus_queue.cc:260] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05fe0af732c49c98184c3bfb1a1015a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } }
I20260806 19:41:02.937518 32165 raft_consensus.cc:388] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:41:02.937741 32165 raft_consensus.cc:482] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:41:02.937989 32165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:02.942243 32165 raft_consensus.cc:504] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05fe0af732c49c98184c3bfb1a1015a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } }
I20260806 19:41:02.943153 32165 leader_election.cc:302] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c05fe0af732c49c98184c3bfb1a1015a; no voters:
I20260806 19:41:02.944595 32165 leader_election.cc:288] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:41:02.944757 32174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:02.945760 32174 raft_consensus.cc:686] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [term 1 LEADER]: Becoming Leader. State: Replica: c05fe0af732c49c98184c3bfb1a1015a, State: Running, Role: LEADER
I20260806 19:41:02.946890 32174 consensus_queue.cc:237] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05fe0af732c49c98184c3bfb1a1015a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } }
I20260806 19:41:02.950544 32174 sys_catalog.cc:455] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c05fe0af732c49c98184c3bfb1a1015a. Latest consensus state: current_term: 1 leader_uuid: "c05fe0af732c49c98184c3bfb1a1015a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05fe0af732c49c98184c3bfb1a1015a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } } }
I20260806 19:41:02.951380 32174 sys_catalog.cc:458] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:02.952096 32176 sys_catalog.cc:455] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c05fe0af732c49c98184c3bfb1a1015a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05fe0af732c49c98184c3bfb1a1015a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42695 } } }
I20260806 19:41:02.952769 32176 sys_catalog.cc:458] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:02.954763 32165 sys_catalog.cc:564] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:02.959668 32179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:41:02.967468 32179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:02.994620 32179 catalog_manager.cc:1347] Generated new cluster ID: c38f96c8cf5443bcadc0ef9dd46fc51a
I20260806 19:41:02.994975 32179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:03.016038 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:02.973909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:51359 (local address 127.0.0.1:42695)
0806 19:41:02.974115 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:02.974121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:02.993261 (+ 19140us) server_negotiation.cc:407] Connection header received
0806 19:41:02.993562 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:02.993569 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:02.993687 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:02.993827 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:02.997447 (+ 3620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:02.999905 (+ 2458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:03.002356 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:03.003131 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:03.003241 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:03.007163 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:03.007171 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:03.007177 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:03.007235 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:03.011726 (+ 4491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:03.012911 (+ 1185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:03.012918 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:03.012922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:03.013111 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:03.013803 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:03.013809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:03.013813 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:03.014337 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:03.014637 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:03.015363 (+ 726us) server_negotiation.cc:299] Negotiation successful
0806 19:41:03.015595 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":74}
I20260806 19:41:03.039438 32179 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:03.041729 32179 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:03.052160 32179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c05fe0af732c49c98184c3bfb1a1015a: Generated new TSK 0
I20260806 19:41:03.053237 32179 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:41:03.066162 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 32098
I20260806 19:41:03.067483 32160 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:03.165279 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 32098
W20260806 19:41:03.187363 32082 connection.cc:535] client connection to 127.0.0.1:42695 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42695): Connection reset by peer (error 104)
2026-08-06T19:41:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2467 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:03 dist-test-slave-ss4n krb5kdc[32199](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:03 dist-test-slave-ss4n krb5kdc[32199](info): set up 2 sockets
Aug 06 19:41:03 dist-test-slave-ss4n krb5kdc[32199](info): commencing operation
krb5kdc: starting...
W20260806 19:41:04.442234 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.006s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:41:04 dist-test-slave-ss4n krb5kdc[32199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:41:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:41:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:41:04.875151 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40019
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:05.303277 32212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:05.303700 32212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:05.303900 32212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:05.313721 32212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:05.313941 32212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:05.314055 32212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:05.314167 32212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:05.314234 32212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:05.326212 32212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:05.327788 32212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:05.329239 32212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:05.343187 32217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:05.343168 32218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:05.343144 32220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:05.344100 32212 server_base.cc:1034] running on GCE node
I20260806 19:41:05.345088 32212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:05.347139 32212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:05.348635 32212 hybrid_clock.cc:648] HybridClock initialized: now 1786045265348579 us; error 49 us; skew 500 ppm
I20260806 19:41:05.349100 32212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:05 dist-test-slave-ss4n krb5kdc[32199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045265, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:05.354281 32212 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:41:05.356335 32212 webserver.cc:460] Webserver started at http://127.0.0.1:46347/ using document root <none> and password file <none>
I20260806 19:41:05.357349 32212 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:05.357656 32212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:05.358042 32212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:05.361546 32212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "cc93bd08cc2c46e8af06752ae28382d3"
format_stamp: "Formatted at 2026-08-06 19:41:05 on dist-test-slave-ss4n"
server_key: "e33210be6c8a9b0c9eb84654aaaf8292"
server_key_iv: "8257cbbfcb14d04646021e117aacd877"
server_key_version: "encryptionkey@0"
I20260806 19:41:05.362480 32212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "cc93bd08cc2c46e8af06752ae28382d3"
format_stamp: "Formatted at 2026-08-06 19:41:05 on dist-test-slave-ss4n"
server_key: "e33210be6c8a9b0c9eb84654aaaf8292"
server_key_iv: "8257cbbfcb14d04646021e117aacd877"
server_key_version: "encryptionkey@0"
I20260806 19:41:05.368716 32212 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260806 19:41:05.374006 32227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:05.376325 32212 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260806 19:41:05.376654 32212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "cc93bd08cc2c46e8af06752ae28382d3"
format_stamp: "Formatted at 2026-08-06 19:41:05 on dist-test-slave-ss4n"
server_key: "e33210be6c8a9b0c9eb84654aaaf8292"
server_key_iv: "8257cbbfcb14d04646021e117aacd877"
server_key_version: "encryptionkey@0"
I20260806 19:41:05.376930 32212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:05.403576 32212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:05.410586 32212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:05.411028 32212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:05.435985 32212 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40019
I20260806 19:41:05.436051 32279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40019 every 8 connection(s)
I20260806 19:41:05.437885 32212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:05.440951 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 32212
I20260806 19:41:05.441179 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:41:05.441917 10340 external_mini_cluster.cc:1428] Setting key c9183a9446a0b126b4926c7e8085a8b8
I20260806 19:41:05.443392 32280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:05.461123 32283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:05.444788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:55003 (local address 127.0.0.1:40019)
0806 19:41:05.445729 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:05.445742 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:05.445788 (+ 46us) server_negotiation.cc:407] Connection header received
0806 19:41:05.447478 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:05.447506 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:05.448054 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:05.448481 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:05.449985 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:05.451964 (+ 1979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:05.454548 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:05.455971 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:05.456192 (+ 221us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:05.457963 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:05.457985 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:05.458009 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:05.458815 (+ 806us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:05.458924 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:05.458958 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:05.459159 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:05.459559 (+ 400us) server_negotiation.cc:299] Negotiation successful
0806 19:41:05.459870 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":421,"threads_started":1}
I20260806 19:41:05.462241 32280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3: Bootstrap starting.
I20260806 19:41:05.467154 32280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:05.468505 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:41:05.468647 32280 log.cc:824] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:05.472028 32280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3: No bootstrap required, opened a new log
I20260806 19:41:05.484992 32283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:05.475007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:60657 (local address 127.0.0.1:40019)
0806 19:41:05.475249 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:05.475256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:05.475634 (+ 378us) server_negotiation.cc:407] Connection header received
0806 19:41:05.475846 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:05.475850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:05.475928 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:05.476026 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:05.476745 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:05.477728 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:05.480049 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:05.480762 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:05.480913 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:05.483513 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:05.483528 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:05.483533 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:05.483569 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:05.483646 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:05.483674 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:05.483803 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:05.484309 (+ 506us) server_negotiation.cc:299] Negotiation successful
0806 19:41:05.484395 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 19:41:05.484479 32280 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc93bd08cc2c46e8af06752ae28382d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } }
I20260806 19:41:05.485706 32280 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:05.485881 32280 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc93bd08cc2c46e8af06752ae28382d3, State: Initialized, Role: FOLLOWER
I20260806 19:41:05.486651 32280 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc93bd08cc2c46e8af06752ae28382d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } }
I20260806 19:41:05.487016 32280 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:41:05.487212 32280 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:41:05.487454 32280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:05.490114 32280 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc93bd08cc2c46e8af06752ae28382d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } }
I20260806 19:41:05.490686 32280 leader_election.cc:302] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc93bd08cc2c46e8af06752ae28382d3; no voters:
I20260806 19:41:05.491357 32280 leader_election.cc:288] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:41:05.491616 32290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:05.492511 32290 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [term 1 LEADER]: Becoming Leader. State: Replica: cc93bd08cc2c46e8af06752ae28382d3, State: Running, Role: LEADER
I20260806 19:41:05.493211 32290 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc93bd08cc2c46e8af06752ae28382d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } }
I20260806 19:41:05.493886 32280 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:05.497107 32291 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc93bd08cc2c46e8af06752ae28382d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc93bd08cc2c46e8af06752ae28382d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } } }
I20260806 19:41:05.497668 32291 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:05.497676 32292 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc93bd08cc2c46e8af06752ae28382d3. Latest consensus state: current_term: 1 leader_uuid: "cc93bd08cc2c46e8af06752ae28382d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc93bd08cc2c46e8af06752ae28382d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } } }
I20260806 19:41:05.498193 32292 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:05.500486 32299 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:41:05.505841 32299 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:05.517328 32299 catalog_manager.cc:1347] Generated new cluster ID: 374c526d95ed4e79abdbb05000cf3d19
I20260806 19:41:05.517589 32299 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:05.543020 32299 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:05.545714 32299 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:05.566895 32299 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc93bd08cc2c46e8af06752ae28382d3: Generated new TSK 0
I20260806 19:41:05.568001 32299 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:41:05.591213 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 32212
W20260806 19:41:05.593144 32275 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.245:55003
W20260806 19:41:05.594553 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 32212: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:41:05.594873 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 32212
W20260806 19:41:05.616822 32197 connection.cc:535] client connection to 127.0.0.1:40019 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40019): Connection reset by peer (error 104)
2026-08-06T19:41:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2432 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:05 dist-test-slave-ss4n krb5kdc[32312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:05 dist-test-slave-ss4n krb5kdc[32312](info): set up 2 sockets
Aug 06 19:41:05 dist-test-slave-ss4n krb5kdc[32312](info): commencing operation
krb5kdc: starting...
W20260806 19:41:06.901000 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:41:07 dist-test-slave-ss4n krb5kdc[32312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:41:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:41:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:41:07.302731 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39155
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:07.723371 32326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:07.723737 32326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:07.723850 32326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:07.732985 32326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:07.733160 32326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:07.733212 32326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:07.733281 32326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:07.733330 32326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:07.746675 32326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:07.748411 32326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:07.750823 32326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:07.768045 32334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:07.767982 32332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:07.768026 32331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:07.771589 32326 server_base.cc:1034] running on GCE node
I20260806 19:41:07.772686 32326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:07.774616 32326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:07.776077 32326 hybrid_clock.cc:648] HybridClock initialized: now 1786045267776014 us; error 61 us; skew 500 ppm
I20260806 19:41:07.776634 32326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:07 dist-test-slave-ss4n krb5kdc[32312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045267, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:07.781075 32326 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:41:07.783260 32326 webserver.cc:460] Webserver started at http://127.0.0.1:35399/ using document root <none> and password file <none>
I20260806 19:41:07.784129 32326 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:07.784292 32326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:07.784628 32326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:07.788121 32326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "69e49755bf11471198fc892bd1032c2e"
format_stamp: "Formatted at 2026-08-06 19:41:07 on dist-test-slave-ss4n"
server_key: "1a4116eb2042ade5fee9d872238633a2"
server_key_iv: "89f464c1fad3d8b85dac96b6c36a5d15"
server_key_version: "encryptionkey@0"
I20260806 19:41:07.789041 32326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "69e49755bf11471198fc892bd1032c2e"
format_stamp: "Formatted at 2026-08-06 19:41:07 on dist-test-slave-ss4n"
server_key: "1a4116eb2042ade5fee9d872238633a2"
server_key_iv: "89f464c1fad3d8b85dac96b6c36a5d15"
server_key_version: "encryptionkey@0"
I20260806 19:41:07.797669 32326 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.011s sys 0.000s
I20260806 19:41:07.804149 32341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:07.806931 32326 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.001s
I20260806 19:41:07.807260 32326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "69e49755bf11471198fc892bd1032c2e"
format_stamp: "Formatted at 2026-08-06 19:41:07 on dist-test-slave-ss4n"
server_key: "1a4116eb2042ade5fee9d872238633a2"
server_key_iv: "89f464c1fad3d8b85dac96b6c36a5d15"
server_key_version: "encryptionkey@0"
I20260806 19:41:07.807566 32326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:07.816448 32326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:07.823282 32326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:07.823688 32326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:07.850946 32326 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39155
I20260806 19:41:07.850966 32393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39155 every 8 connection(s)
I20260806 19:41:07.852758 32326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:07.856768 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 32326
I20260806 19:41:07.857008 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:41:07.857667 10340 external_mini_cluster.cc:1428] Setting key 306b3cc10a6887cfd4c3f25809ac1988
I20260806 19:41:07.858235 32394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:07.867097 32397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:07.859865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:42057 (local address 127.0.0.1:39155)
0806 19:41:07.860774 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:07.860787 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:07.860833 (+ 46us) server_negotiation.cc:407] Connection header received
0806 19:41:07.862558 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:07.862585 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:07.863057 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:07.863467 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:07.864856 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:07.864864 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:07.864883 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:07.864914 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:07.865006 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:07.865042 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:07.865266 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:07.865620 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 19:41:07.865905 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":488,"threads_started":1}
I20260806 19:41:07.872571 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:41:07.874881 32394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e: Bootstrap starting.
I20260806 19:41:07.878965 32394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:07.880535 32394 log.cc:824] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:07.881769 32397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:07.878269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:59727 (local address 127.0.0.1:39155)
0806 19:41:07.878524 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:07.878542 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:07.878949 (+ 407us) server_negotiation.cc:407] Connection header received
0806 19:41:07.879241 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:07.879245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:07.879300 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:07.879401 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:07.880915 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:07.880919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:07.880922 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:07.880944 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:07.880992 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:07.881009 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:07.881144 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:07.881311 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 19:41:07.881402 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 19:41:07.884389 32394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e: No bootstrap required, opened a new log
I20260806 19:41:07.893651 32394 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e49755bf11471198fc892bd1032c2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39155 } }
I20260806 19:41:07.894636 32394 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:07.894757 32394 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69e49755bf11471198fc892bd1032c2e, State: Initialized, Role: FOLLOWER
I20260806 19:41:07.895390 32394 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e49755bf11471198fc892bd1032c2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39155 } }
I20260806 19:41:07.895677 32394 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:41:07.895799 32394 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:41:07.895946 32394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:07.899755 32394 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e49755bf11471198fc892bd1032c2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39155 } }
I20260806 19:41:07.900929 32394 leader_election.cc:302] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69e49755bf11471198fc892bd1032c2e; no voters:
I20260806 19:41:07.902002 32394 leader_election.cc:288] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:41:07.902324 32404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:07.903357 32404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [term 1 LEADER]: Becoming Leader. State: Replica: 69e49755bf11471198fc892bd1032c2e, State: Running, Role: LEADER
I20260806 19:41:07.904261 32404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e49755bf11471198fc892bd1032c2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39155 } }
I20260806 19:41:07.905848 32394 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:07.908087 32405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69e49755bf11471198fc892bd1032c2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e49755bf11471198fc892bd1032c2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39155 } } }
I20260806 19:41:07.908748 32405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:07.908118 32406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69e49755bf11471198fc892bd1032c2e. Latest consensus state: current_term: 1 leader_uuid: "69e49755bf11471198fc892bd1032c2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e49755bf11471198fc892bd1032c2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39155 } } }
I20260806 19:41:07.909873 32406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:07.911955 32412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:41:07.938308 32412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:07.949605 32412 catalog_manager.cc:1347] Generated new cluster ID: a5dfd6f5f1c5497cb750962dddb13fa6
I20260806 19:41:07.949777 32412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:07.972553 32412 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:07.973091 32412 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:07.981568 32412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69e49755bf11471198fc892bd1032c2e: Generated new TSK 0
I20260806 19:41:07.982621 32412 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:41:07.997014 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 32326
W20260806 19:41:07.998817 32389 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.245:42057
W20260806 19:41:07.999998 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 32326: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:41:08.000208 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 32326
2026-08-06T19:41:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2400 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:08 dist-test-slave-ss4n krb5kdc[32425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:08 dist-test-slave-ss4n krb5kdc[32425](info): set up 2 sockets
Aug 06 19:41:08 dist-test-slave-ss4n krb5kdc[32425](info): commencing operation
krb5kdc: starting...
W20260806 19:41:09.310330 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.014s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:41:09 dist-test-slave-ss4n krb5kdc[32425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:41:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:41:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
W20260806 19:41:09.778460 10340 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.0.0.1: real 0.107s user 0.004s sys 0.031s
I20260806 19:41:09.779490 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44629
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:10.227319 32438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:10.227665 32438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:10.227802 32438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:10.235608 32438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:10.235755 32438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:10.235839 32438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:10.235954 32438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:10.236017 32438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:10.247344 32438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44629
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:10.248837 32438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:10.250291 32438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:10.264850 32443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:10.264731 32444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:10.264801 32446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:10.266503 32438 server_base.cc:1034] running on GCE node
I20260806 19:41:10.267652 32438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:10.269732 32438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:10.271102 32438 hybrid_clock.cc:648] HybridClock initialized: now 1786045270271017 us; error 55 us; skew 500 ppm
I20260806 19:41:10.271867 32438 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:10 dist-test-slave-ss4n krb5kdc[32425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045270, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:10.277371 32438 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:41:10.279351 32438 webserver.cc:460] Webserver started at http://127.0.0.1:46517/ using document root <none> and password file <none>
I20260806 19:41:10.280215 32438 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:10.280346 32438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:10.280712 32438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:10.284420 32438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "00fec0435c31467793eee9038d590756"
format_stamp: "Formatted at 2026-08-06 19:41:10 on dist-test-slave-ss4n"
server_key: "2a502e86acaca62336a2452f2e7b6ea4"
server_key_iv: "00d79e2fb950f560f68a8247b66b01e9"
server_key_version: "encryptionkey@0"
I20260806 19:41:10.285413 32438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "00fec0435c31467793eee9038d590756"
format_stamp: "Formatted at 2026-08-06 19:41:10 on dist-test-slave-ss4n"
server_key: "2a502e86acaca62336a2452f2e7b6ea4"
server_key_iv: "00d79e2fb950f560f68a8247b66b01e9"
server_key_version: "encryptionkey@0"
I20260806 19:41:10.292196 32438 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 19:41:10.297317 32453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:10.299237 32438 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:41:10.299494 32438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "00fec0435c31467793eee9038d590756"
format_stamp: "Formatted at 2026-08-06 19:41:10 on dist-test-slave-ss4n"
server_key: "2a502e86acaca62336a2452f2e7b6ea4"
server_key_iv: "00d79e2fb950f560f68a8247b66b01e9"
server_key_version: "encryptionkey@0"
I20260806 19:41:10.299733 32438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:10.315389 32438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:10.321327 32438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:10.321821 32438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:10.345263 32438 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44629
I20260806 19:41:10.345323 32506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44629 every 8 connection(s)
I20260806 19:41:10.347083 32438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:10.351737 32507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:10.353780 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 32438
I20260806 19:41:10.353958 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:41:10.354740 10340 external_mini_cluster.cc:1428] Setting key 007a04ac86868c091c886f050451448e
I20260806 19:41:10.372059 32507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756: Bootstrap starting.
Aug 06 19:41:10 dist-test-slave-ss4n krb5kdc[32425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 19:41:10.376377 32507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:10.378093 32507 log.cc:824] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:10.382239 32507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756: No bootstrap required, opened a new log
I20260806 19:41:10.387521 32510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:10.357573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:56161 (local address 127.0.0.1:44629)
0806 19:41:10.358785 (+ 1212us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:10.358804 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:10.358859 (+ 55us) server_negotiation.cc:407] Connection header received
0806 19:41:10.361890 (+ 3031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:10.361932 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:10.362650 (+ 718us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:10.363198 (+ 548us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:10.365190 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:10.367459 (+ 2269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:10.369076 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:10.370962 (+ 1886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:10.371277 (+ 315us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:10.376285 (+ 5008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:10.376299 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:10.376326 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:10.376400 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:10.381246 (+ 4846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:10.382512 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:10.382524 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:10.382534 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:10.382814 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:10.383704 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:10.383709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:10.383713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:10.384527 (+ 814us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:10.384904 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:10.385553 (+ 649us) server_negotiation.cc:299] Negotiation successful
0806 19:41:10.385983 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":929,"thread_start_us":468,"threads_started":1}
I20260806 19:41:10.393267 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:41:10.392947 32507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00fec0435c31467793eee9038d590756" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44629 } }
I20260806 19:41:10.393553 32507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:10.393671 32507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00fec0435c31467793eee9038d590756, State: Initialized, Role: FOLLOWER
I20260806 19:41:10.394398 32507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00fec0435c31467793eee9038d590756" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44629 } }
I20260806 19:41:10.394711 32507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:41:10.394858 32507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:41:10.395042 32507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:10.399281 32507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00fec0435c31467793eee9038d590756" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44629 } }
I20260806 19:41:10.400099 32507 leader_election.cc:302] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00fec0435c31467793eee9038d590756; no voters:
I20260806 19:41:10.400959 32507 leader_election.cc:288] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:41:10.401185 32517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:10.404114 32507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:10.404115 32517 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [term 1 LEADER]: Becoming Leader. State: Replica: 00fec0435c31467793eee9038d590756, State: Running, Role: LEADER
I20260806 19:41:10.405520 32517 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00fec0435c31467793eee9038d590756" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44629 } }
I20260806 19:41:10.412247 32518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00fec0435c31467793eee9038d590756" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00fec0435c31467793eee9038d590756" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44629 } } }
I20260806 19:41:10.414345 32518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:10.413460 32519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00fec0435c31467793eee9038d590756. Latest consensus state: current_term: 1 leader_uuid: "00fec0435c31467793eee9038d590756" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00fec0435c31467793eee9038d590756" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44629 } } }
I20260806 19:41:10.415365 32519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:10.415727 32525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:41:10.423578 32525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:10.428684 32510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:10.399657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:47589 (local address 127.0.0.1:44629)
0806 19:41:10.399889 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:10.399896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:10.400590 (+ 694us) server_negotiation.cc:407] Connection header received
0806 19:41:10.400827 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:10.400837 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:10.400964 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:10.401113 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:10.403644 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:10.405318 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:10.410308 (+ 4990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:10.411699 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:10.411930 (+ 231us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:10.416999 (+ 5069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:10.417008 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:10.417016 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:10.417065 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:10.423136 (+ 6071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:10.424631 (+ 1495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:10.424641 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:10.424645 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:10.424882 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:10.425701 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:10.425707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:10.425711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:10.426186 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:10.426512 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:10.427901 (+ 1389us) server_negotiation.cc:299] Negotiation successful
0806 19:41:10.428259 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 19:41:10.444068 32525 catalog_manager.cc:1347] Generated new cluster ID: 3c3cadb726c644edbd14e3c80de1d513
I20260806 19:41:10.444269 32525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:10.461522 32525 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:10.463904 32525 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:10.477977 32525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00fec0435c31467793eee9038d590756: Generated new TSK 0
I20260806 19:41:10.479063 32525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:41:10.494107 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 32438
I20260806 19:41:10.496620 32502 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:10.598980 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 32438
W20260806 19:41:10.620443 32423 connection.cc:535] client connection to 127.0.0.1:44629 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44629): Connection reset by peer (error 104)
2026-08-06T19:41:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2599 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:10 dist-test-slave-ss4n krb5kdc[32540](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:10 dist-test-slave-ss4n krb5kdc[32540](info): set up 2 sockets
Aug 06 19:41:10 dist-test-slave-ss4n krb5kdc[32540](info): commencing operation
krb5kdc: starting...
W20260806 19:41:11.864624 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.003s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:41:12 dist-test-slave-ss4n krb5kdc[32540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:41:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:41:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:41:12.234161 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35073
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:12.655527 32553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:12.656103 32553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:12.656316 32553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:12.665418 32553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:12.665704 32553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:12.665812 32553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:12.665921 32553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:12.666013 32553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:12.678776 32553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35073
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:12.680379 32553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:12.681965 32553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:12.696553 32561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:12.696681 32558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:12.696655 32559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:12.697365 32553 server_base.cc:1034] running on GCE node
I20260806 19:41:12.699046 32553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:12.701978 32553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:12.703445 32553 hybrid_clock.cc:648] HybridClock initialized: now 1786045272703346 us; error 75 us; skew 500 ppm
I20260806 19:41:12.704054 32553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:12 dist-test-slave-ss4n krb5kdc[32540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045272, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:12.708575 32553 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:41:12.710779 32553 webserver.cc:460] Webserver started at http://127.0.0.1:34419/ using document root <none> and password file <none>
I20260806 19:41:12.711743 32553 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:12.711971 32553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:12.712518 32553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:12.716619 32553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "679aae85424c46fb8702aa39eded96a7"
format_stamp: "Formatted at 2026-08-06 19:41:12 on dist-test-slave-ss4n"
server_key: "1bbd8d8ecf9c1744ff99c2fc60d9d556"
server_key_iv: "6986950b227d095f16b0688d01e1c1a7"
server_key_version: "encryptionkey@0"
I20260806 19:41:12.717679 32553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "679aae85424c46fb8702aa39eded96a7"
format_stamp: "Formatted at 2026-08-06 19:41:12 on dist-test-slave-ss4n"
server_key: "1bbd8d8ecf9c1744ff99c2fc60d9d556"
server_key_iv: "6986950b227d095f16b0688d01e1c1a7"
server_key_version: "encryptionkey@0"
I20260806 19:41:12.724299 32553 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 19:41:12.729067 32569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:12.730844 32553 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:41:12.731103 32553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "679aae85424c46fb8702aa39eded96a7"
format_stamp: "Formatted at 2026-08-06 19:41:12 on dist-test-slave-ss4n"
server_key: "1bbd8d8ecf9c1744ff99c2fc60d9d556"
server_key_iv: "6986950b227d095f16b0688d01e1c1a7"
server_key_version: "encryptionkey@0"
I20260806 19:41:12.731359 32553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:12.754591 32553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:12.760354 32553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:12.760751 32553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:12.786075 32553 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35073
I20260806 19:41:12.786077 32622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35073 every 8 connection(s)
I20260806 19:41:12.788218 32553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:12.793143 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 32553
I20260806 19:41:12.793373 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:41:12.793754 32623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:12.794065 10340 external_mini_cluster.cc:1428] Setting key 3197a7a4e5b63d6ed5b3e8d64af3ff7c
I20260806 19:41:12.810424 32623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7: Bootstrap starting.
I20260806 19:41:12.810811 32626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:12.796228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:54973 (local address 127.0.0.1:35073)
0806 19:41:12.797122 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:12.797138 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:12.797186 (+ 48us) server_negotiation.cc:407] Connection header received
0806 19:41:12.799511 (+ 2325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:12.799543 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:12.800120 (+ 577us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:12.800568 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:12.801378 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:12.803219 (+ 1841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:12.805246 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:12.806128 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:12.806300 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:12.807891 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:12.807900 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:12.807921 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:12.808621 (+ 700us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:12.808720 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:12.808752 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:12.808891 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:12.809356 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 19:41:12.809664 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":420,"threads_started":1}
I20260806 19:41:12.813969 32623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:12.815317 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:41:12.815261 32623 log.cc:824] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:12.819316 32623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7: No bootstrap required, opened a new log
I20260806 19:41:12.829892 32626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:12.821101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:48073 (local address 127.0.0.1:35073)
0806 19:41:12.821319 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:12.821326 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:12.821885 (+ 559us) server_negotiation.cc:407] Connection header received
0806 19:41:12.822098 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:12.822104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:12.822210 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:12.822341 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:12.822908 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:12.824134 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:12.826473 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:12.827238 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:12.827389 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:12.828922 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:12.828929 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:12.828934 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:12.828961 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:12.829022 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:12.829047 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:12.829183 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:12.829422 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 19:41:12.829529 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 19:41:12.829757 32623 raft_consensus.cc:348] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679aae85424c46fb8702aa39eded96a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35073 } }
I20260806 19:41:12.830533 32623 raft_consensus.cc:374] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:12.830758 32623 raft_consensus.cc:729] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 679aae85424c46fb8702aa39eded96a7, State: Initialized, Role: FOLLOWER
I20260806 19:41:12.831907 32623 consensus_queue.cc:260] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679aae85424c46fb8702aa39eded96a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35073 } }
I20260806 19:41:12.832413 32623 raft_consensus.cc:388] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:41:12.832645 32623 raft_consensus.cc:482] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:41:12.832858 32623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:12.835644 32623 raft_consensus.cc:504] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679aae85424c46fb8702aa39eded96a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35073 } }
I20260806 19:41:12.836287 32623 leader_election.cc:302] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 679aae85424c46fb8702aa39eded96a7; no voters:
I20260806 19:41:12.837127 32623 leader_election.cc:288] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:41:12.837282 32633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:12.838286 32633 raft_consensus.cc:686] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [term 1 LEADER]: Becoming Leader. State: Replica: 679aae85424c46fb8702aa39eded96a7, State: Running, Role: LEADER
I20260806 19:41:12.838913 32633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679aae85424c46fb8702aa39eded96a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35073 } }
I20260806 19:41:12.839732 32623 sys_catalog.cc:564] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:12.842630 32635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 679aae85424c46fb8702aa39eded96a7. Latest consensus state: current_term: 1 leader_uuid: "679aae85424c46fb8702aa39eded96a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679aae85424c46fb8702aa39eded96a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35073 } } }
I20260806 19:41:12.843168 32635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:12.843740 32634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "679aae85424c46fb8702aa39eded96a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679aae85424c46fb8702aa39eded96a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35073 } } }
I20260806 19:41:12.844058 32634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:12.846457 32642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:41:12.852366 32642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:12.863945 32642 catalog_manager.cc:1347] Generated new cluster ID: 3cf93071bfd74f028ed9b5de3a378148
I20260806 19:41:12.864117 32642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:12.886054 32642 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:12.887616 32642 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:12.898272 32642 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 679aae85424c46fb8702aa39eded96a7: Generated new TSK 0
I20260806 19:41:12.899365 32642 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:41:12.911613 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 32553
W20260806 19:41:12.913003 32617 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.245:54973
W20260806 19:41:12.913996 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 32553: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:41:12.914194 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 32553
W20260806 19:41:12.933977 32538 connection.cc:535] client connection to 127.0.0.1:35073 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35073): Connection reset by peer (error 104)
2026-08-06T19:41:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:13 dist-test-slave-ss4n krb5kdc[32654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:13 dist-test-slave-ss4n krb5kdc[32654](info): set up 2 sockets
Aug 06 19:41:13 dist-test-slave-ss4n krb5kdc[32654](info): commencing operation
krb5kdc: starting...
W20260806 19:41:14.193168 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:41:14 dist-test-slave-ss4n krb5kdc[32654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:41:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:41:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 19:41:14.562654 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36105
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:14.968540 32667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:14.969255 32667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:14.969532 32667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:14.977973 32667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:14.978185 32667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:14.978308 32667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:14.978420 32667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:14.978499 32667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:14.989203 32667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36105
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:14.990779 32667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:14.992188 32667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:15.005743 32673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:15.005746 32676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:15.005770 32674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:15.006799 32667 server_base.cc:1034] running on GCE node
I20260806 19:41:15.007742 32667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:15.009425 32667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:15.010777 32667 hybrid_clock.cc:648] HybridClock initialized: now 1786045275010720 us; error 42 us; skew 500 ppm
I20260806 19:41:15.011260 32667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:15 dist-test-slave-ss4n krb5kdc[32654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045275, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:15.015985 32667 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 19:41:15.018414 32667 webserver.cc:460] Webserver started at http://127.0.0.1:41081/ using document root <none> and password file <none>
I20260806 19:41:15.019243 32667 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:15.019416 32667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:15.019770 32667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:15.023304 32667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "86b7c41fa2cd440b9f5298e47e89356d"
format_stamp: "Formatted at 2026-08-06 19:41:15 on dist-test-slave-ss4n"
server_key: "08ebcc6929a027afb6c1d162b0422c78"
server_key_iv: "74798718fde55af710d0cde04c2994c3"
server_key_version: "encryptionkey@0"
I20260806 19:41:15.024217 32667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "86b7c41fa2cd440b9f5298e47e89356d"
format_stamp: "Formatted at 2026-08-06 19:41:15 on dist-test-slave-ss4n"
server_key: "08ebcc6929a027afb6c1d162b0422c78"
server_key_iv: "74798718fde55af710d0cde04c2994c3"
server_key_version: "encryptionkey@0"
I20260806 19:41:15.030435 32667 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260806 19:41:15.035561 32683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:15.037741 32667 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 19:41:15.038004 32667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "86b7c41fa2cd440b9f5298e47e89356d"
format_stamp: "Formatted at 2026-08-06 19:41:15 on dist-test-slave-ss4n"
server_key: "08ebcc6929a027afb6c1d162b0422c78"
server_key_iv: "74798718fde55af710d0cde04c2994c3"
server_key_version: "encryptionkey@0"
I20260806 19:41:15.038276 32667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:15.046159 32667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:15.051715 32667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:15.052106 32667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:15.076300 32667 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36105
I20260806 19:41:15.076404 32735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36105 every 8 connection(s)
I20260806 19:41:15.078017 32667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:15.080022 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 32667
I20260806 19:41:15.080308 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:41:15.080987 10340 external_mini_cluster.cc:1428] Setting key 22c1e643038a0d859cebfb489a680652
I20260806 19:41:15.082877 32736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:15.092012 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:15.083448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:38897 (local address 127.0.0.1:36105)
0806 19:41:15.084392 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:15.084410 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:15.084481 (+ 71us) server_negotiation.cc:407] Connection header received
0806 19:41:15.086778 (+ 2297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:15.086814 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:15.087379 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:15.087956 (+ 577us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:15.089520 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:15.089537 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:15.089563 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:15.089588 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:15.089702 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:15.089754 (+ 52us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:15.090057 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:15.090230 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 19:41:15.090499 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":377,"threads_started":1}
I20260806 19:41:15.097259 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:41:15.100057 32736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d: Bootstrap starting.
I20260806 19:41:15.104360 32736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:15.105897 32736 log.cc:824] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:15.106104 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:15.102245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:38703 (local address 127.0.0.1:36105)
0806 19:41:15.102451 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:15.102460 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:15.102893 (+ 433us) server_negotiation.cc:407] Connection header received
0806 19:41:15.103410 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:15.103417 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:15.103508 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:15.103649 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:15.105129 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:15.105135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:15.105139 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:15.105162 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:15.105242 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:15.105262 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:15.105385 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:15.105635 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 19:41:15.105701 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 19:41:15.109205 32736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d: No bootstrap required, opened a new log
I20260806 19:41:15.119565 32736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b7c41fa2cd440b9f5298e47e89356d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36105 } }
I20260806 19:41:15.120061 32736 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:15.120193 32736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86b7c41fa2cd440b9f5298e47e89356d, State: Initialized, Role: FOLLOWER
I20260806 19:41:15.120922 32736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b7c41fa2cd440b9f5298e47e89356d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36105 } }
I20260806 19:41:15.121212 32736 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:41:15.121363 32736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:41:15.121649 32736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:15.124688 32736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b7c41fa2cd440b9f5298e47e89356d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36105 } }
I20260806 19:41:15.125471 32736 leader_election.cc:302] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86b7c41fa2cd440b9f5298e47e89356d; no voters:
I20260806 19:41:15.126318 32736 leader_election.cc:288] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:41:15.126507 32747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:15.127400 32747 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [term 1 LEADER]: Becoming Leader. State: Replica: 86b7c41fa2cd440b9f5298e47e89356d, State: Running, Role: LEADER
I20260806 19:41:15.128046 32747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b7c41fa2cd440b9f5298e47e89356d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36105 } }
I20260806 19:41:15.129153 32736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:15.131093 32749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86b7c41fa2cd440b9f5298e47e89356d. Latest consensus state: current_term: 1 leader_uuid: "86b7c41fa2cd440b9f5298e47e89356d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b7c41fa2cd440b9f5298e47e89356d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36105 } } }
I20260806 19:41:15.131675 32749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:15.131079 32748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86b7c41fa2cd440b9f5298e47e89356d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b7c41fa2cd440b9f5298e47e89356d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36105 } } }
I20260806 19:41:15.132041 32748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:15.135438 32753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:41:15.141327 32753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:15.154225 32753 catalog_manager.cc:1347] Generated new cluster ID: 72ca7fef10f94b3eb4da16d9ae3f2eb1
I20260806 19:41:15.154469 32753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:15.192567 32753 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:15.193158 32753 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:15.202426 32753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86b7c41fa2cd440b9f5298e47e89356d: Generated new TSK 0
I20260806 19:41:15.203490 32753 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:41:15.218536 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 32667
W20260806 19:41:15.220359 32731 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.245:38897
W20260806 19:41:15.221407 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 32667: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:41:15.221664 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 32667
2026-08-06T19:41:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2309 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:15 dist-test-slave-ss4n krb5kdc[301](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:15 dist-test-slave-ss4n krb5kdc[301](info): set up 2 sockets
Aug 06 19:41:15 dist-test-slave-ss4n krb5kdc[301](info): commencing operation
krb5kdc: starting...
W20260806 19:41:16.476435 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.008s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:41:16 dist-test-slave-ss4n krb5kdc[301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:41:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:41:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:41:16.883668 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:46823
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:46823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:17.298663 314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:17.299026 314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:17.299229 314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:17.307039 314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:17.307180 314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:17.307229 314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:17.307351 314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:17.307431 314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:17.318219 314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:46823
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:46823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:17.319692 314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:17.321187 314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:17.337441 319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:17.337368 322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:17.337281 320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:17.338613 314 server_base.cc:1034] running on GCE node
I20260806 19:41:17.339861 314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:17.341861 314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:17.343168 314 hybrid_clock.cc:648] HybridClock initialized: now 1786045277343105 us; error 66 us; skew 500 ppm
I20260806 19:41:17.343706 314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:17 dist-test-slave-ss4n krb5kdc[301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045277, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:17.353277 314 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:41:17.355400 314 webserver.cc:460] Webserver started at http://127.10.25.62:39307/ using document root <none> and password file <none>
I20260806 19:41:17.356283 314 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:17.356415 314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:17.356833 314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:17.360517 314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "48a7cd3c839941978236e196393a7806"
format_stamp: "Formatted at 2026-08-06 19:41:17 on dist-test-slave-ss4n"
server_key: "2f5613aba1fc66b9f25e28d090663b43"
server_key_iv: "d1e81a77a841aea93c4e97162fedcafc"
server_key_version: "encryptionkey@0"
I20260806 19:41:17.361613 314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "48a7cd3c839941978236e196393a7806"
format_stamp: "Formatted at 2026-08-06 19:41:17 on dist-test-slave-ss4n"
server_key: "2f5613aba1fc66b9f25e28d090663b43"
server_key_iv: "d1e81a77a841aea93c4e97162fedcafc"
server_key_version: "encryptionkey@0"
I20260806 19:41:17.369037 314 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 19:41:17.373677 329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:17.375895 314 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:41:17.376260 314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "48a7cd3c839941978236e196393a7806"
format_stamp: "Formatted at 2026-08-06 19:41:17 on dist-test-slave-ss4n"
server_key: "2f5613aba1fc66b9f25e28d090663b43"
server_key_iv: "d1e81a77a841aea93c4e97162fedcafc"
server_key_version: "encryptionkey@0"
I20260806 19:41:17.376616 314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:17.403839 314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:17.408978 314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:17.409298 314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:17.431833 314 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:46823
I20260806 19:41:17.431880 380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:46823 every 8 connection(s)
I20260806 19:41:17.433552 314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:17.436278 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 314
I20260806 19:41:17.436551 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:41:17.437096 10340 external_mini_cluster.cc:1428] Setting key 057c39818bd64c93d87402faba4c1169
I20260806 19:41:17.438584 381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 19:41:17 dist-test-slave-ss4n krb5kdc[301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:41:17.459556 381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806: Bootstrap starting.
I20260806 19:41:17.463500 381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:17.464818 381 log.cc:824] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:17.468398 381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806: No bootstrap required, opened a new log
I20260806 19:41:17.470850 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:17.440380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39257 (local address 127.10.25.62:46823)
0806 19:41:17.442320 (+ 1940us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:17.442338 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:17.442839 (+ 501us) server_negotiation.cc:407] Connection header received
0806 19:41:17.444981 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:17.445011 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:17.445693 (+ 682us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:17.446141 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:17.448733 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:17.450432 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:17.453289 (+ 2857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:17.454272 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:17.454460 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:17.461251 (+ 6791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:17.461267 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:17.461283 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:17.461343 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:17.465364 (+ 4021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:17.466596 (+ 1232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:17.466605 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:17.466613 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:17.466832 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:17.467440 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:17.467444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:17.467446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:17.468037 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:17.468443 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:17.469232 (+ 789us) server_negotiation.cc:299] Negotiation successful
0806 19:41:17.469659 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1425,"spinlock_wait_cycles":25472,"thread_start_us":1329,"threads_started":1}
I20260806 19:41:17.476858 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:41:17.479120 381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a7cd3c839941978236e196393a7806" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 46823 } }
I20260806 19:41:17.479660 381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:17.479789 381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48a7cd3c839941978236e196393a7806, State: Initialized, Role: FOLLOWER
I20260806 19:41:17.480646 381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a7cd3c839941978236e196393a7806" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 46823 } }
I20260806 19:41:17.481096 381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:41:17.481297 381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:41:17.481675 381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:17.485044 381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a7cd3c839941978236e196393a7806" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 46823 } }
I20260806 19:41:17.485772 381 leader_election.cc:302] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48a7cd3c839941978236e196393a7806; no voters:
I20260806 19:41:17.486552 381 leader_election.cc:288] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:41:17.486749 391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:17.488416 391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [term 1 LEADER]: Becoming Leader. State: Replica: 48a7cd3c839941978236e196393a7806, State: Running, Role: LEADER
I20260806 19:41:17.489269 391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a7cd3c839941978236e196393a7806" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 46823 } }
I20260806 19:41:17.490479 381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:17.493188 392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48a7cd3c839941978236e196393a7806" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a7cd3c839941978236e196393a7806" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 46823 } } }
I20260806 19:41:17.493741 392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:17.497208 393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48a7cd3c839941978236e196393a7806. Latest consensus state: current_term: 1 leader_uuid: "48a7cd3c839941978236e196393a7806" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a7cd3c839941978236e196393a7806" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 46823 } } }
I20260806 19:41:17.497958 393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:17.501003 396 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:41:17.507117 396 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:17.510690 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:17.482919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32859 (local address 127.10.25.62:46823)
0806 19:41:17.483137 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:17.483142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:17.483710 (+ 568us) server_negotiation.cc:407] Connection header received
0806 19:41:17.483970 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:17.483982 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:17.484058 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:17.484181 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:17.485954 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:17.487360 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:17.489987 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:17.490967 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:17.492282 (+ 1315us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:17.496936 (+ 4654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:17.496944 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:17.496949 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:17.497037 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:17.503115 (+ 6078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:17.505004 (+ 1889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:17.505015 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:17.505022 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:17.505337 (+ 315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:17.506365 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:17.506374 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:17.506381 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:17.507029 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:17.507472 (+ 443us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:17.510102 (+ 2630us) server_negotiation.cc:299] Negotiation successful
0806 19:41:17.510390 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 19:41:17.520758 396 catalog_manager.cc:1347] Generated new cluster ID: 161b5fc438af433fa45d5a535aad6237
I20260806 19:41:17.520901 396 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:17.553236 396 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:17.555198 396 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:17.564203 396 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48a7cd3c839941978236e196393a7806: Generated new TSK 0
I20260806 19:41:17.565472 396 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:41:17.581146 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 314
I20260806 19:41:17.583556 376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:17.680123 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 314
W20260806 19:41:17.701067 32767 connection.cc:535] client connection to 127.10.25.62:46823 recv error: Network error: failed to read from TLS socket (remote: 127.10.25.62:46823): Connection reset by peer (error 104)
2026-08-06T19:41:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:17 dist-test-slave-ss4n krb5kdc[415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:17 dist-test-slave-ss4n krb5kdc[415](info): set up 2 sockets
Aug 06 19:41:17 dist-test-slave-ss4n krb5kdc[415](info): commencing operation
krb5kdc: starting...
W20260806 19:41:18.941146 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:41:19 dist-test-slave-ss4n krb5kdc[415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:41:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:41:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:41:19.299154 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:42557
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:46277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:42557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:19.687727 429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:19.688077 429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:19.688254 429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:19.697510 429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:19.697711 429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:19.697814 429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:19.697919 429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:19.697999 429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:19.711160 429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:46277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:42557
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:42557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:19.712723 429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:19.714270 429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:19.727923 434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:19.728021 435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:19.727986 437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:19.729188 429 server_base.cc:1034] running on GCE node
I20260806 19:41:19.730261 429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:19.732105 429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:19.733419 429 hybrid_clock.cc:648] HybridClock initialized: now 1786045279733351 us; error 56 us; skew 500 ppm
I20260806 19:41:19.733901 429 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:19 dist-test-slave-ss4n krb5kdc[415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045279, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:19.738466 429 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:41:19.740609 429 webserver.cc:460] Webserver started at http://127.10.25.62:37051/ using document root <none> and password file <none>
I20260806 19:41:19.741875 429 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:19.742049 429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:19.742478 429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:19.745949 429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "65440c68311d4519a39521e9542f4735"
format_stamp: "Formatted at 2026-08-06 19:41:19 on dist-test-slave-ss4n"
server_key: "87e5e45761b4d169e3b771c4a34ded15"
server_key_iv: "f71751a5ada66df222ee4bc545e0a282"
server_key_version: "encryptionkey@0"
I20260806 19:41:19.746886 429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "65440c68311d4519a39521e9542f4735"
format_stamp: "Formatted at 2026-08-06 19:41:19 on dist-test-slave-ss4n"
server_key: "87e5e45761b4d169e3b771c4a34ded15"
server_key_iv: "f71751a5ada66df222ee4bc545e0a282"
server_key_version: "encryptionkey@0"
I20260806 19:41:19.753717 429 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 19:41:19.758819 444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:19.760874 429 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:41:19.761086 429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "65440c68311d4519a39521e9542f4735"
format_stamp: "Formatted at 2026-08-06 19:41:19 on dist-test-slave-ss4n"
server_key: "87e5e45761b4d169e3b771c4a34ded15"
server_key_iv: "f71751a5ada66df222ee4bc545e0a282"
server_key_version: "encryptionkey@0"
I20260806 19:41:19.761385 429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:19.785790 429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:19.790911 429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:19.791270 429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:19.816578 429 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:42557
I20260806 19:41:19.816603 496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:42557 every 8 connection(s)
I20260806 19:41:19.818419 429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:19.823129 497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:19.827000 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 429
I20260806 19:41:19.827268 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:41:19.828011 10340 external_mini_cluster.cc:1428] Setting key adcfce7d4b9efb43c99d5bee8967c73f
I20260806 19:41:19.839777 497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735: Bootstrap starting.
I20260806 19:41:19.843675 497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:19.844786 500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:19.831345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41913 (local address 127.10.25.62:42557)
0806 19:41:19.832406 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:19.832421 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:19.832472 (+ 51us) server_negotiation.cc:407] Connection header received
0806 19:41:19.834602 (+ 2130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:19.834629 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:19.835137 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:19.835590 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:19.836270 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:19.837798 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:19.839307 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:19.840304 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:19.840562 (+ 258us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:19.842060 (+ 1498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:19.842069 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:19.842084 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:19.842775 (+ 691us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:19.842881 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:19.842910 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:19.843113 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:19.843425 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 19:41:19.843626 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":452,"threads_started":1}
I20260806 19:41:19.845140 497 log.cc:824] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:19.848896 497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735: No bootstrap required, opened a new log
I20260806 19:41:19.849669 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:41:19.858808 497 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65440c68311d4519a39521e9542f4735" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 42557 } }
I20260806 19:41:19.859434 497 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:19.859625 497 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65440c68311d4519a39521e9542f4735, State: Initialized, Role: FOLLOWER
I20260806 19:41:19.860363 497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65440c68311d4519a39521e9542f4735" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 42557 } }
I20260806 19:41:19.860630 497 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:41:19.860836 497 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:41:19.861030 497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:19.864602 497 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65440c68311d4519a39521e9542f4735" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 42557 } }
I20260806 19:41:19.865406 500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:19.855625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54577 (local address 127.10.25.62:42557)
0806 19:41:19.855938 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:19.855946 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:19.856578 (+ 632us) server_negotiation.cc:407] Connection header received
0806 19:41:19.857012 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:19.857019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:19.857125 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:19.857274 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:19.858047 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:19.859300 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:19.861782 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:19.862679 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:19.862863 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:19.864185 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:19.864195 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:19.864203 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:19.864234 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:19.864291 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:19.864321 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:19.864474 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:19.864954 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 19:41:19.865048 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260806 19:41:19.865360 497 leader_election.cc:302] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65440c68311d4519a39521e9542f4735; no voters:
I20260806 19:41:19.866307 497 leader_election.cc:288] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:41:19.866443 507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:19.868209 507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [term 1 LEADER]: Becoming Leader. State: Replica: 65440c68311d4519a39521e9542f4735, State: Running, Role: LEADER
I20260806 19:41:19.868916 507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65440c68311d4519a39521e9542f4735" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 42557 } }
I20260806 19:41:19.869364 497 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:19.872704 508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65440c68311d4519a39521e9542f4735" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65440c68311d4519a39521e9542f4735" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 42557 } } }
I20260806 19:41:19.873178 508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:19.874127 509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65440c68311d4519a39521e9542f4735. Latest consensus state: current_term: 1 leader_uuid: "65440c68311d4519a39521e9542f4735" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65440c68311d4519a39521e9542f4735" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 42557 } } }
I20260806 19:41:19.874641 509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:19.875586 512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:41:19.881654 512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:19.896723 512 catalog_manager.cc:1347] Generated new cluster ID: 5dddd0e2e7474cfe9a5e55ece5f06015
I20260806 19:41:19.896871 512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:19.928539 512 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:19.930133 512 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:19.941249 512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65440c68311d4519a39521e9542f4735: Generated new TSK 0
I20260806 19:41:19.942385 512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:41:19.958623 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 429
W20260806 19:41:19.960130 492 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41913
W20260806 19:41:19.961149 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 429: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:41:19.961330 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 429
W20260806 19:41:19.980878 413 connection.cc:535] client connection to 127.10.25.62:42557 recv error: Network error: failed to read from TLS socket (remote: 127.10.25.62:42557): Connection reset by peer (error 104)
2026-08-06T19:41:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2279 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:20 dist-test-slave-ss4n krb5kdc[529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:20 dist-test-slave-ss4n krb5kdc[529](info): set up 2 sockets
Aug 06 19:41:20 dist-test-slave-ss4n krb5kdc[529](info): commencing operation
krb5kdc: starting...
W20260806 19:41:21.218587 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:41:21 dist-test-slave-ss4n krb5kdc[529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:41:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:41:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:41:21.602252 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:40133
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:33453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:40133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:22.015148 543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:22.015482 543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:22.015646 543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:22.023609 543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:22.023767 543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:22.023857 543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:22.023979 543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:22.024049 543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:22.034789 543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:33453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:40133
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:40133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:22.036278 543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:22.037916 543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:22.052646 549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:22.052891 551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:22.053464 543 server_base.cc:1034] running on GCE node
W20260806 19:41:22.053555 548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:22.054600 543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:22.056347 543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:22.057777 543 hybrid_clock.cc:648] HybridClock initialized: now 1786045282057677 us; error 102 us; skew 500 ppm
I20260806 19:41:22.058445 543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:22 dist-test-slave-ss4n krb5kdc[529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045282, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:22.062990 543 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:41:22.065675 543 webserver.cc:460] Webserver started at http://127.10.25.62:32807/ using document root <none> and password file <none>
I20260806 19:41:22.067090 543 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:22.067447 543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:22.067992 543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:22.071902 543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "423f0b3b7e9c470ab5f72cb3d54b080f"
format_stamp: "Formatted at 2026-08-06 19:41:22 on dist-test-slave-ss4n"
server_key: "f99a589e49843d1e6e029047a29324c1"
server_key_iv: "71cff2fe3b23beccfce654875b7c8469"
server_key_version: "encryptionkey@0"
I20260806 19:41:22.072968 543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "423f0b3b7e9c470ab5f72cb3d54b080f"
format_stamp: "Formatted at 2026-08-06 19:41:22 on dist-test-slave-ss4n"
server_key: "f99a589e49843d1e6e029047a29324c1"
server_key_iv: "71cff2fe3b23beccfce654875b7c8469"
server_key_version: "encryptionkey@0"
I20260806 19:41:22.079483 543 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260806 19:41:22.084076 558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:22.085847 543 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:41:22.086097 543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "423f0b3b7e9c470ab5f72cb3d54b080f"
format_stamp: "Formatted at 2026-08-06 19:41:22 on dist-test-slave-ss4n"
server_key: "f99a589e49843d1e6e029047a29324c1"
server_key_iv: "71cff2fe3b23beccfce654875b7c8469"
server_key_version: "encryptionkey@0"
I20260806 19:41:22.086376 543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:22.093731 543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:22.099761 543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:22.100198 543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:22.123775 543 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:40133
I20260806 19:41:22.123827 611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:40133 every 8 connection(s)
I20260806 19:41:22.125406 543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:22.129938 612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:22.133965 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 543
I20260806 19:41:22.134197 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:41:22.134824 10340 external_mini_cluster.cc:1428] Setting key d3b072b463ae17344428ba6d88b90eeb
I20260806 19:41:22.144011 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:22.137152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43495 (local address 127.10.25.62:40133)
0806 19:41:22.138190 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:22.138204 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:22.138252 (+ 48us) server_negotiation.cc:407] Connection header received
0806 19:41:22.140062 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:22.140090 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:22.140496 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:22.140981 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:22.142263 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:22.142270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:22.142288 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:22.142304 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:22.142366 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:22.142388 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:22.142653 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:22.142801 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 19:41:22.143010 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":509,"threads_started":1}
I20260806 19:41:22.144711 612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f: Bootstrap starting.
I20260806 19:41:22.148644 612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:22.149158 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:41:22.150127 612 log.cc:824] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:22.154119 612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f: No bootstrap required, opened a new log
I20260806 19:41:22.158294 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:22.154949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33625 (local address 127.10.25.62:40133)
0806 19:41:22.155201 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:22.155210 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:22.155567 (+ 357us) server_negotiation.cc:407] Connection header received
0806 19:41:22.155802 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:22.155806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:22.155859 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:22.155976 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:22.157446 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:22.157452 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:22.157466 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:22.157545 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:22.157606 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:22.157625 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:22.157786 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:22.157924 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 19:41:22.157996 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 19:41:22.164810 612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "423f0b3b7e9c470ab5f72cb3d54b080f" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40133 } }
I20260806 19:41:22.165320 612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:22.165455 612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 423f0b3b7e9c470ab5f72cb3d54b080f, State: Initialized, Role: FOLLOWER
I20260806 19:41:22.166277 612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "423f0b3b7e9c470ab5f72cb3d54b080f" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40133 } }
I20260806 19:41:22.166580 612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:41:22.166751 612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:41:22.166985 612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:22.170030 612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "423f0b3b7e9c470ab5f72cb3d54b080f" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40133 } }
I20260806 19:41:22.170670 612 leader_election.cc:302] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 423f0b3b7e9c470ab5f72cb3d54b080f; no voters:
I20260806 19:41:22.171525 612 leader_election.cc:288] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:41:22.171792 622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:22.172763 622 raft_consensus.cc:686] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [term 1 LEADER]: Becoming Leader. State: Replica: 423f0b3b7e9c470ab5f72cb3d54b080f, State: Running, Role: LEADER
I20260806 19:41:22.173596 622 consensus_queue.cc:237] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "423f0b3b7e9c470ab5f72cb3d54b080f" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40133 } }
I20260806 19:41:22.174706 612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:22.177577 623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "423f0b3b7e9c470ab5f72cb3d54b080f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "423f0b3b7e9c470ab5f72cb3d54b080f" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40133 } } }
I20260806 19:41:22.178208 623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:22.179047 624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 423f0b3b7e9c470ab5f72cb3d54b080f. Latest consensus state: current_term: 1 leader_uuid: "423f0b3b7e9c470ab5f72cb3d54b080f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "423f0b3b7e9c470ab5f72cb3d54b080f" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 40133 } } }
I20260806 19:41:22.179477 629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:41:22.179507 624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:22.187795 629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:22.201218 629 catalog_manager.cc:1347] Generated new cluster ID: c25a482f39a84558844c37b3577456bc
I20260806 19:41:22.201434 629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:22.213671 629 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:22.214385 629 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:22.223500 629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 423f0b3b7e9c470ab5f72cb3d54b080f: Generated new TSK 0
I20260806 19:41:22.224556 629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:41:22.233035 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 543
W20260806 19:41:22.234392 607 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43495
W20260806 19:41:22.235355 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 543: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:41:22.235541 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 543
2026-08-06T19:41:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2276 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:22 dist-test-slave-ss4n krb5kdc[643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:22 dist-test-slave-ss4n krb5kdc[643](info): set up 2 sockets
Aug 06 19:41:22 dist-test-slave-ss4n krb5kdc[643](info): commencing operation
krb5kdc: starting...
W20260806 19:41:23.480523 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.006s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:41:23 dist-test-slave-ss4n krb5kdc[643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:41:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:41:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:41:23.850939 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:43957
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:43031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:43957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:24.250774 657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:24.251187 657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:24.251370 657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:24.258942 657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:24.259145 657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:24.259265 657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:24.259384 657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:24.259476 657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:24.269869 657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:43031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:43957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:43957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:24.271361 657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:24.272797 657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:24.286389 662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:24.286437 665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:24.286500 663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:24.291990 657 server_base.cc:1034] running on GCE node
I20260806 19:41:24.293032 657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:24.295013 657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:24.296288 657 hybrid_clock.cc:648] HybridClock initialized: now 1786045284296225 us; error 63 us; skew 500 ppm
I20260806 19:41:24.296774 657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:24 dist-test-slave-ss4n krb5kdc[643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045284, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:24.304710 657 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:41:24.306914 657 webserver.cc:460] Webserver started at http://127.10.25.62:46757/ using document root <none> and password file <none>
I20260806 19:41:24.307832 657 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:24.308023 657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:24.308400 657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:24.313772 657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "c68af3b0817645ee99715cf79e2588cc"
format_stamp: "Formatted at 2026-08-06 19:41:24 on dist-test-slave-ss4n"
server_key: "8311479eb70fe8fa8ad515e467910ccd"
server_key_iv: "d75bb68c3a772530632e080b7e358450"
server_key_version: "encryptionkey@0"
I20260806 19:41:24.314962 657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "c68af3b0817645ee99715cf79e2588cc"
format_stamp: "Formatted at 2026-08-06 19:41:24 on dist-test-slave-ss4n"
server_key: "8311479eb70fe8fa8ad515e467910ccd"
server_key_iv: "d75bb68c3a772530632e080b7e358450"
server_key_version: "encryptionkey@0"
I20260806 19:41:24.322275 657 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260806 19:41:24.327646 672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:24.330302 657 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260806 19:41:24.330590 657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "c68af3b0817645ee99715cf79e2588cc"
format_stamp: "Formatted at 2026-08-06 19:41:24 on dist-test-slave-ss4n"
server_key: "8311479eb70fe8fa8ad515e467910ccd"
server_key_iv: "d75bb68c3a772530632e080b7e358450"
server_key_version: "encryptionkey@0"
I20260806 19:41:24.330856 657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:24.368942 657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:24.375092 657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:24.375520 657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:24.400441 657 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:43957
I20260806 19:41:24.400475 724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:43957 every 8 connection(s)
I20260806 19:41:24.402233 657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:24.406921 725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:24.409212 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 657
I20260806 19:41:24.409409 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:41:24.409895 10340 external_mini_cluster.cc:1428] Setting key a93b6db49d25c2d0a0ff3fce4dbb26e7
I20260806 19:41:24.421168 725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc: Bootstrap starting.
I20260806 19:41:24.425872 725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc: Neither blocks nor log segments found. Creating new log.
Aug 06 19:41:24 dist-test-slave-ss4n krb5kdc[643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:41:24.427310 725 log.cc:824] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:24.431751 725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc: No bootstrap required, opened a new log
I20260806 19:41:24.440387 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:24.412071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36399 (local address 127.10.25.62:43957)
0806 19:41:24.413067 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:24.413081 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:24.413133 (+ 52us) server_negotiation.cc:407] Connection header received
0806 19:41:24.415187 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:24.415219 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:24.415753 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:24.416172 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:24.418249 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:24.419800 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:24.421899 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:24.422772 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:24.422972 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:24.429226 (+ 6254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:24.429244 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:24.429270 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:24.429330 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:24.434412 (+ 5082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:24.435528 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:24.435537 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:24.435545 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:24.435777 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:24.436339 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:24.436345 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:24.436349 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:24.437196 (+ 847us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:24.437610 (+ 414us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:24.438170 (+ 560us) server_negotiation.cc:299] Negotiation successful
0806 19:41:24.438720 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":471,"threads_started":1}
I20260806 19:41:24.443405 725 raft_consensus.cc:348] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68af3b0817645ee99715cf79e2588cc" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43957 } }
I20260806 19:41:24.444005 725 raft_consensus.cc:374] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:24.444151 725 raft_consensus.cc:729] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c68af3b0817645ee99715cf79e2588cc, State: Initialized, Role: FOLLOWER
I20260806 19:41:24.445060 725 consensus_queue.cc:260] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68af3b0817645ee99715cf79e2588cc" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43957 } }
I20260806 19:41:24.445344 725 raft_consensus.cc:388] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:41:24.445590 725 raft_consensus.cc:482] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:41:24.445820 725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:24.446070 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:41:24.448596 725 raft_consensus.cc:504] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68af3b0817645ee99715cf79e2588cc" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43957 } }
I20260806 19:41:24.449291 725 leader_election.cc:302] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c68af3b0817645ee99715cf79e2588cc; no voters:
I20260806 19:41:24.450207 725 leader_election.cc:288] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:41:24.450344 731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:24.452322 731 raft_consensus.cc:686] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [term 1 LEADER]: Becoming Leader. State: Replica: c68af3b0817645ee99715cf79e2588cc, State: Running, Role: LEADER
I20260806 19:41:24.453753 725 sys_catalog.cc:564] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:24.453890 731 consensus_queue.cc:237] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68af3b0817645ee99715cf79e2588cc" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43957 } }
I20260806 19:41:24.457574 735 sys_catalog.cc:455] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c68af3b0817645ee99715cf79e2588cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68af3b0817645ee99715cf79e2588cc" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43957 } } }
I20260806 19:41:24.458129 735 sys_catalog.cc:458] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:24.459133 736 sys_catalog.cc:455] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c68af3b0817645ee99715cf79e2588cc. Latest consensus state: current_term: 1 leader_uuid: "c68af3b0817645ee99715cf79e2588cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c68af3b0817645ee99715cf79e2588cc" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43957 } } }
I20260806 19:41:24.459729 736 sys_catalog.cc:458] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:24.464923 741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:41:24.481622 741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:24.487936 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:24.453032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55709 (local address 127.10.25.62:43957)
0806 19:41:24.453294 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:24.453308 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:24.453704 (+ 396us) server_negotiation.cc:407] Connection header received
0806 19:41:24.454078 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:24.454087 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:24.454201 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:24.454296 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:24.457732 (+ 3436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:24.459458 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:24.462653 (+ 3195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:24.464112 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:24.464364 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:24.469142 (+ 4778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:24.469154 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:24.469163 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:24.469235 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:24.482409 (+ 13174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:24.483717 (+ 1308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:24.483725 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:24.483734 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:24.483927 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:24.484536 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:24.484541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:24.484544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:24.485379 (+ 835us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:24.486999 (+ 1620us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:24.487187 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 19:41:24.487472 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 19:41:24.494813 741 catalog_manager.cc:1347] Generated new cluster ID: dec37cd9b04f467c9982c60ef26e3b2c
I20260806 19:41:24.495014 741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:24.518296 741 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:24.520378 741 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:24.539731 741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c68af3b0817645ee99715cf79e2588cc: Generated new TSK 0
I20260806 19:41:24.541291 741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:41:24.549045 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 657
I20260806 19:41:24.550813 720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:24.646409 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 657
2026-08-06T19:41:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2408 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:24 dist-test-slave-ss4n krb5kdc[757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:24 dist-test-slave-ss4n krb5kdc[757](info): set up 2 sockets
Aug 06 19:41:24 dist-test-slave-ss4n krb5kdc[757](info): commencing operation
krb5kdc: starting...
W20260806 19:41:25.898627 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.003s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:41:26 dist-test-slave-ss4n krb5kdc[757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:41:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:41:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:41:26.270574 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:46035
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:46159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:46035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:26.651165 770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:26.651516 770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:26.651676 770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:26.659502 770 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:26.659639 770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:26.659765 770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:26.659862 770 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:26.659904 770 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:26.670509 770 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:46159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:46035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:46035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:26.672008 770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:26.673422 770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:26.686705 778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:26.686642 776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:26.686671 775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:26.688433 770 server_base.cc:1034] running on GCE node
I20260806 19:41:26.689625 770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:26.691473 770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:26.692813 770 hybrid_clock.cc:648] HybridClock initialized: now 1786045286692768 us; error 75 us; skew 500 ppm
I20260806 19:41:26.693277 770 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:26 dist-test-slave-ss4n krb5kdc[757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045286, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:26.697446 770 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:41:26.699687 770 webserver.cc:460] Webserver started at http://127.10.25.62:36711/ using document root <none> and password file <none>
I20260806 19:41:26.700464 770 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:26.700603 770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:26.700886 770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:26.704371 770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "929e1e8045054639ac4390d66d5eb784"
format_stamp: "Formatted at 2026-08-06 19:41:26 on dist-test-slave-ss4n"
server_key: "97b7b0de106b8ad7f7f5e14a93bb39cd"
server_key_iv: "daa68b1ede201e19b527f62ad75d22ca"
server_key_version: "encryptionkey@0"
I20260806 19:41:26.705253 770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "929e1e8045054639ac4390d66d5eb784"
format_stamp: "Formatted at 2026-08-06 19:41:26 on dist-test-slave-ss4n"
server_key: "97b7b0de106b8ad7f7f5e14a93bb39cd"
server_key_iv: "daa68b1ede201e19b527f62ad75d22ca"
server_key_version: "encryptionkey@0"
I20260806 19:41:26.711796 770 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260806 19:41:26.717111 786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:26.719203 770 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 19:41:26.719408 770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "929e1e8045054639ac4390d66d5eb784"
format_stamp: "Formatted at 2026-08-06 19:41:26 on dist-test-slave-ss4n"
server_key: "97b7b0de106b8ad7f7f5e14a93bb39cd"
server_key_iv: "daa68b1ede201e19b527f62ad75d22ca"
server_key_version: "encryptionkey@0"
I20260806 19:41:26.719733 770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:26.736583 770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:26.742116 770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:26.742504 770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:26.765179 770 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:46035
I20260806 19:41:26.765224 838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:46035 every 8 connection(s)
I20260806 19:41:26.766801 770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:26.770592 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 770
I20260806 19:41:26.770812 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:41:26.771229 839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:26.771374 10340 external_mini_cluster.cc:1428] Setting key bd9d9af43a41a0fddddfcb60b99113e7
I20260806 19:41:26.787416 839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784: Bootstrap starting.
I20260806 19:41:26.787734 842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:26.773466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59239 (local address 127.10.25.62:46035)
0806 19:41:26.774537 (+ 1071us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:26.774551 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:26.774603 (+ 52us) server_negotiation.cc:407] Connection header received
0806 19:41:26.776506 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:26.776535 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:26.777045 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:26.777462 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:26.778250 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:26.779791 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:26.782122 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:26.782924 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:26.783096 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:26.784771 (+ 1675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:26.784784 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:26.784811 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:26.785670 (+ 859us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:26.785767 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:26.785797 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:26.785897 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:26.786261 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 19:41:26.786601 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":459,"threads_started":1}
I20260806 19:41:26.792399 839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:26.792822 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:41:26.793908 839 log.cc:824] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:26.798152 839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784: No bootstrap required, opened a new log
I20260806 19:41:26.807962 842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:26.798590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40945 (local address 127.10.25.62:46035)
0806 19:41:26.798881 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:26.798893 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:26.799489 (+ 596us) server_negotiation.cc:407] Connection header received
0806 19:41:26.799650 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:26.799655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:26.799770 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:26.799920 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:26.800705 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:26.801758 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:26.803966 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:26.804871 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:26.805064 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:26.806917 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:26.806923 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:26.806927 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:26.806956 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:26.807021 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:26.807040 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:26.807148 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:26.807394 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 19:41:26.807510 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 19:41:26.811398 839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929e1e8045054639ac4390d66d5eb784" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 46035 } }
I20260806 19:41:26.811918 839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:26.812043 839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929e1e8045054639ac4390d66d5eb784, State: Initialized, Role: FOLLOWER
I20260806 19:41:26.812680 839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929e1e8045054639ac4390d66d5eb784" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 46035 } }
I20260806 19:41:26.812983 839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:41:26.813171 839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:41:26.813372 839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:26.815804 839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929e1e8045054639ac4390d66d5eb784" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 46035 } }
I20260806 19:41:26.816406 839 leader_election.cc:302] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 929e1e8045054639ac4390d66d5eb784; no voters:
I20260806 19:41:26.817065 839 leader_election.cc:288] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:41:26.817245 849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:26.818125 849 raft_consensus.cc:686] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [term 1 LEADER]: Becoming Leader. State: Replica: 929e1e8045054639ac4390d66d5eb784, State: Running, Role: LEADER
I20260806 19:41:26.818857 849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929e1e8045054639ac4390d66d5eb784" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 46035 } }
I20260806 19:41:26.819828 839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:26.821817 850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "929e1e8045054639ac4390d66d5eb784" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929e1e8045054639ac4390d66d5eb784" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 46035 } } }
I20260806 19:41:26.822302 850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:26.822880 851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 929e1e8045054639ac4390d66d5eb784. Latest consensus state: current_term: 1 leader_uuid: "929e1e8045054639ac4390d66d5eb784" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "929e1e8045054639ac4390d66d5eb784" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 46035 } } }
I20260806 19:41:26.823690 851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:26.824090 857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:41:26.838536 857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:26.851140 857 catalog_manager.cc:1347] Generated new cluster ID: 6ef77c7a48d4468db9369ad9cdb7a187
I20260806 19:41:26.851338 857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:26.861395 857 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:26.862926 857 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:26.872056 857 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 929e1e8045054639ac4390d66d5eb784: Generated new TSK 0
I20260806 19:41:26.873025 857 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:41:26.880834 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 770
W20260806 19:41:26.882100 834 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59239
W20260806 19:41:26.882865 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 770: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:41:26.883026 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 770
2026-08-06T19:41:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2235 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:27 dist-test-slave-ss4n krb5kdc[871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:27 dist-test-slave-ss4n krb5kdc[871](info): set up 2 sockets
Aug 06 19:41:27 dist-test-slave-ss4n krb5kdc[871](info): commencing operation
krb5kdc: starting...
W20260806 19:41:28.150272 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.004s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:41:28 dist-test-slave-ss4n krb5kdc[871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:41:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:41:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:41:28.514901 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:38201
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:38201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:28.900260 885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:28.900589 885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:28.900686 885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:28.908442 885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:28.908584 885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:28.908638 885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:28.908691 885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:28.908737 885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:28.919289 885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:38201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:38201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:28.920703 885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:28.922096 885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:28.935642 890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:28.936030 893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:28.936023 891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:28.937162 885 server_base.cc:1034] running on GCE node
I20260806 19:41:28.938230 885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:28.939898 885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:28.941174 885 hybrid_clock.cc:648] HybridClock initialized: now 1786045288941118 us; error 46 us; skew 500 ppm
I20260806 19:41:28.941737 885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:28 dist-test-slave-ss4n krb5kdc[871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045288, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:28.945664 885 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:41:28.947834 885 webserver.cc:460] Webserver started at http://127.10.25.62:39473/ using document root <none> and password file <none>
I20260806 19:41:28.949002 885 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:28.949152 885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:28.949788 885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:28.953313 885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "a795ab9eb02040f5874ea087458ef7d6"
format_stamp: "Formatted at 2026-08-06 19:41:28 on dist-test-slave-ss4n"
server_key: "c212e30c13c27c3450c8c5ee7419c178"
server_key_iv: "97a79839c628e53c30f0090c58a8efc0"
server_key_version: "encryptionkey@0"
I20260806 19:41:28.954279 885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "a795ab9eb02040f5874ea087458ef7d6"
format_stamp: "Formatted at 2026-08-06 19:41:28 on dist-test-slave-ss4n"
server_key: "c212e30c13c27c3450c8c5ee7419c178"
server_key_iv: "97a79839c628e53c30f0090c58a8efc0"
server_key_version: "encryptionkey@0"
I20260806 19:41:28.960489 885 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 19:41:28.965967 900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:28.968216 885 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260806 19:41:28.968583 885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "a795ab9eb02040f5874ea087458ef7d6"
format_stamp: "Formatted at 2026-08-06 19:41:28 on dist-test-slave-ss4n"
server_key: "c212e30c13c27c3450c8c5ee7419c178"
server_key_iv: "97a79839c628e53c30f0090c58a8efc0"
server_key_version: "encryptionkey@0"
I20260806 19:41:28.968890 885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:28.976975 885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:28.982971 885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:28.983342 885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:29.007901 885 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:38201
I20260806 19:41:29.007913 951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:38201 every 8 connection(s)
I20260806 19:41:29.009722 885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:29.012635 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 885
I20260806 19:41:29.012818 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:41:29.014072 10340 external_mini_cluster.cc:1428] Setting key e838c92639e8561e7ae2efc45e33eb52
I20260806 19:41:29.014322 952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:29.023677 955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:29.016731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56857 (local address 127.10.25.62:38201)
0806 19:41:29.017825 (+ 1094us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:29.017838 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:29.017889 (+ 51us) server_negotiation.cc:407] Connection header received
0806 19:41:29.019571 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:29.019600 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:29.020036 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:29.020547 (+ 511us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:29.021788 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:29.021794 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:29.021808 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:29.021825 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:29.021889 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:29.021912 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:29.022221 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:29.022386 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 19:41:29.022618 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":436,"threads_started":1}
I20260806 19:41:29.028692 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:41:29.030397 952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6: Bootstrap starting.
I20260806 19:41:29.034164 952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:29.035707 952 log.cc:824] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:29.036906 955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:29.033814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39059 (local address 127.10.25.62:38201)
0806 19:41:29.034093 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:29.034105 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:29.034634 (+ 529us) server_negotiation.cc:407] Connection header received
0806 19:41:29.034848 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:29.034853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:29.034910 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:29.035009 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:29.036083 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:29.036087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:29.036091 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 19:41:29.036108 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:29.036154 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 19:41:29.036172 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:29.036302 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:29.036483 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 19:41:29.036572 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 19:41:29.039332 952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6: No bootstrap required, opened a new log
I20260806 19:41:29.047802 952 raft_consensus.cc:348] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a795ab9eb02040f5874ea087458ef7d6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38201 } }
I20260806 19:41:29.048391 952 raft_consensus.cc:374] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 19:41:29.048591 952 raft_consensus.cc:729] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a795ab9eb02040f5874ea087458ef7d6, State: Initialized, Role: FOLLOWER
I20260806 19:41:29.049333 952 consensus_queue.cc:260] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a795ab9eb02040f5874ea087458ef7d6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38201 } }
I20260806 19:41:29.049669 952 raft_consensus.cc:388] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 19:41:29.049820 952 raft_consensus.cc:482] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 19:41:29.050002 952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:29.052419 952 raft_consensus.cc:504] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a795ab9eb02040f5874ea087458ef7d6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38201 } }
I20260806 19:41:29.053273 952 leader_election.cc:302] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a795ab9eb02040f5874ea087458ef7d6; no voters:
I20260806 19:41:29.054533 952 leader_election.cc:288] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 19:41:29.054735 962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:29.055905 962 raft_consensus.cc:686] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [term 1 LEADER]: Becoming Leader. State: Replica: a795ab9eb02040f5874ea087458ef7d6, State: Running, Role: LEADER
I20260806 19:41:29.056715 962 consensus_queue.cc:237] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a795ab9eb02040f5874ea087458ef7d6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38201 } }
I20260806 19:41:29.057175 952 sys_catalog.cc:564] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:29.060490 964 sys_catalog.cc:455] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a795ab9eb02040f5874ea087458ef7d6. Latest consensus state: current_term: 1 leader_uuid: "a795ab9eb02040f5874ea087458ef7d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a795ab9eb02040f5874ea087458ef7d6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38201 } } }
I20260806 19:41:29.060387 963 sys_catalog.cc:455] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a795ab9eb02040f5874ea087458ef7d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a795ab9eb02040f5874ea087458ef7d6" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 38201 } } }
I20260806 19:41:29.060921 963 sys_catalog.cc:458] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:29.060921 964 sys_catalog.cc:458] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:29.062952 970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:41:29.069224 970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:29.080506 970 catalog_manager.cc:1347] Generated new cluster ID: f61e5182a822474983e6416a5e2b4f53
I20260806 19:41:29.080664 970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:29.109109 970 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:29.109748 970 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:29.118076 970 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a795ab9eb02040f5874ea087458ef7d6: Generated new TSK 0
I20260806 19:41:29.119086 970 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 19:41:29.125867 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 885
W20260806 19:41:29.127326 947 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56857
W20260806 19:41:29.128203 10340 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 885: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 19:41:29.128386 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 885
2026-08-06T19:41:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2242 ms)
[----------] 18 tests from AuthTokenIssuingTest (42596 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:29 dist-test-slave-ss4n krb5kdc[984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:29 dist-test-slave-ss4n krb5kdc[984](info): set up 2 sockets
Aug 06 19:41:29 dist-test-slave-ss4n krb5kdc[984](info): commencing operation
krb5kdc: starting...
W20260806 19:41:30.413733 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.003s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:41:30 dist-test-slave-ss4n krb5kdc[984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:41:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:41:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:41:30.786890 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:35051
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:40487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:35051,127.10.25.61:43017,127.10.25.60:38053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:31.183075 998 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 19:41:31.183559 998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:31.183701 998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:31.183841 998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:31.191762 998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:31.191946 998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:31.192065 998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:31.192173 998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:31.192298 998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:31.205279 998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:40487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:35051,127.10.25.61:43017,127.10.25.60:38053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:35051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:31.207074 998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:31.208604 998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:31.223274 1003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:31.223630 1004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:31.223765 1006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:31.225647 998 server_base.cc:1034] running on GCE node
I20260806 19:41:31.227039 998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:31.229204 998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:31.230574 998 hybrid_clock.cc:648] HybridClock initialized: now 1786045291230428 us; error 89 us; skew 500 ppm
I20260806 19:41:31.231050 998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:31 dist-test-slave-ss4n krb5kdc[984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045291, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:31.240839 998 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:41:31.243098 998 webserver.cc:460] Webserver started at http://127.10.25.62:41735/ using document root <none> and password file <none>
I20260806 19:41:31.243949 998 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:31.244100 998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:31.244463 998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:31.248092 998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "663b4999aec244bf86ee3c94e1c8d08a"
format_stamp: "Formatted at 2026-08-06 19:41:31 on dist-test-slave-ss4n"
server_key: "0293b3ab2dab75de5092f98514f9297d"
server_key_iv: "89e221eee687d189b4bafc597a847d6a"
server_key_version: "encryptionkey@0"
I20260806 19:41:31.248988 998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "663b4999aec244bf86ee3c94e1c8d08a"
format_stamp: "Formatted at 2026-08-06 19:41:31 on dist-test-slave-ss4n"
server_key: "0293b3ab2dab75de5092f98514f9297d"
server_key_iv: "89e221eee687d189b4bafc597a847d6a"
server_key_version: "encryptionkey@0"
I20260806 19:41:31.257406 998 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260806 19:41:31.262560 1013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:31.264344 998 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 19:41:31.264629 998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "663b4999aec244bf86ee3c94e1c8d08a"
format_stamp: "Formatted at 2026-08-06 19:41:31 on dist-test-slave-ss4n"
server_key: "0293b3ab2dab75de5092f98514f9297d"
server_key_iv: "89e221eee687d189b4bafc597a847d6a"
server_key_version: "encryptionkey@0"
I20260806 19:41:31.264947 998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:31.311632 998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:31.317006 998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:31.317392 998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:31.340929 998 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:35051
I20260806 19:41:31.340953 1069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:35051 every 8 connection(s)
I20260806 19:41:31.342948 998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:31.347858 1070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:31.349383 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 998
I20260806 19:41:31.349689 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:41:31.350634 10340 external_mini_cluster.cc:1428] Setting key 28b9998107815ff47ab8d3af3ed30357
I20260806 19:41:31.360002 1070 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:41:31 dist-test-slave-ss4n krb5kdc[984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045291, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.25.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.61@KRBTEST.COM" created.
I20260806 19:41:31.388114 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:31.366444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49060 (local address 127.10.25.62:35051)
0806 19:41:31.367389 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:31.367399 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:31.367459 (+ 60us) server_negotiation.cc:407] Connection header received
0806 19:41:31.369071 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:31.369078 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:31.369409 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:31.369714 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:31.371320 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:31.372735 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:31.374454 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:31.375116 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:31.375360 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:31.379424 (+ 4064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:31.379434 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:31.379451 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:31.379486 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:31.382855 (+ 3369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:31.383687 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:31.383695 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:31.383697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:31.383878 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:31.384459 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:31.384463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:31.384465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:31.385069 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:31.385330 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:31.386004 (+ 674us) server_negotiation.cc:299] Negotiation successful
0806 19:41:31.386386 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":512,"threads_started":1}
I20260806 19:41:31.388141 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:31.364868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:35051 (local address 127.0.0.1:49060)
0806 19:41:31.365991 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:31.366058 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:31.368557 (+ 2499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:31.369729 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:31.369737 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:31.370677 (+ 940us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:31.371068 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:31.371089 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:31.372926 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:31.372933 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:31.374280 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:31.374295 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:31.375394 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:31.375402 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:31.375637 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:31.376442 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:31.376470 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:31.379119 (+ 2649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:31.383035 (+ 3916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:31.383050 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:31.383064 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:31.383512 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:31.384067 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:31.384075 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:31.384080 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:31.384314 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:31.385240 (+ 926us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:31.385255 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:31.385548 (+ 293us) client_negotiation.cc:769] Sending connection context
0806 19:41:31.385852 (+ 304us) client_negotiation.cc:240] Negotiation successful
0806 19:41:31.386559 (+ 707us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"mutex_wait_us":175,"thread_start_us":432,"threads_started":1}
I20260806 19:41:31.392063 1070 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:41:31.393556 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:31.392974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:43017 (local address 127.0.0.1:53952)
0806 19:41:31.393220 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:31.393354 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260806 19:41:31.393934 1016 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.25.61:43017: connect: Connection refused (error 111)
I20260806 19:41:31.396268 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:31.395575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:43017 (local address 127.0.0.1:53956)
0806 19:41:31.395951 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:31.396062 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260806 19:41:31.396587 1070 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.61:43017: Network error: Client connection negotiation failed: client connection to 127.10.25.61:43017: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.61.keytab.
Entry for principal kudu/127.10.25.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.61.keytab.
I20260806 19:41:31.435364 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.61:43017
--webserver_interface=127.10.25.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.61.keytab
--principal=kudu/127.10.25.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:40487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:35051,127.10.25.61:43017,127.10.25.60:38053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:41:31.446229 1070 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } attempt: 1
I20260806 19:41:31.447582 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:31.447050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:43017 (local address 127.0.0.1:53966)
0806 19:41:31.447275 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:31.447376 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 19:41:31.448971 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:31.448491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:43017 (local address 127.0.0.1:53974)
0806 19:41:31.448696 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:31.448790 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260806 19:41:31.449268 1070 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.61:43017: Network error: Client connection negotiation failed: client connection to 127.10.25.61:43017: connect: Connection refused (error 111)
I20260806 19:41:31.517763 1070 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } attempt: 2
I20260806 19:41:31.519307 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:31.518761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:43017 (local address 127.0.0.1:53984)
0806 19:41:31.519006 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:31.519124 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 19:41:31.521092 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:31.520485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:43017 (local address 127.0.0.1:54000)
0806 19:41:31.520751 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:31.520893 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260806 19:41:31.521667 1070 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.61:43017: Network error: Client connection negotiation failed: client connection to 127.10.25.61:43017: connect: Connection refused (error 111)
I20260806 19:41:31.613000 1070 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } attempt: 3
I20260806 19:41:31.614367 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:31.613905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:43017 (local address 127.0.0.1:54016)
0806 19:41:31.614125 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:31.614221 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 19:41:31.615947 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:31.615150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:43017 (local address 127.0.0.1:54018)
0806 19:41:31.615532 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:31.615678 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260806 19:41:31.616443 1070 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.61:43017: Network error: Client connection negotiation failed: client connection to 127.10.25.61:43017: connect: Connection refused (error 111)
I20260806 19:41:31.759819 1070 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } attempt: 4
I20260806 19:41:31.761332 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:31.760567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:43017 (local address 127.0.0.1:54028)
0806 19:41:31.760937 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:31.761094 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199}
I20260806 19:41:31.762537 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:31.762092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:43017 (local address 127.0.0.1:54036)
0806 19:41:31.762288 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:31.762396 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260806 19:41:31.762794 1070 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.61:43017: Network error: Client connection negotiation failed: client connection to 127.10.25.61:43017: connect: Connection refused (error 111)
W20260806 19:41:31.861615 1078 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 19:41:31.862242 1078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:31.862689 1078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:31.862933 1078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:31.874356 1078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:31.874607 1078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:31.874747 1078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:31.874877 1078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:31.875043 1078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:31.888653 1078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:40487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:35051,127.10.25.61:43017,127.10.25.60:38053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.61.keytab
--principal=kudu/127.10.25.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.61:43017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.25.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:31.890345 1078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:31.892053 1078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:31.906123 1083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:31.906713 1086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:31.906745 1084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:31.907948 1078 server_base.cc:1034] running on GCE node
I20260806 19:41:31.908921 1078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:31.910790 1078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:31.912119 1078 hybrid_clock.cc:648] HybridClock initialized: now 1786045291912050 us; error 38 us; skew 500 ppm
I20260806 19:41:31.912546 1078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:31 dist-test-slave-ss4n krb5kdc[984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045291, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:31.917855 1078 init.cc:377] Logged in from keytab as kudu/127.10.25.61@KRBTEST.COM (short username kudu)
I20260806 19:41:31.920764 1078 webserver.cc:460] Webserver started at http://127.10.25.61:35143/ using document root <none> and password file <none>
I20260806 19:41:31.921876 1078 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:31.922022 1078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:31.922482 1078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:31.926497 1078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/data/instance:
uuid: "7745fb9c574442bf8450a52661be4c35"
format_stamp: "Formatted at 2026-08-06 19:41:31 on dist-test-slave-ss4n"
server_key: "5303eaa8b7901f5cf042b2728aa28f42"
server_key_iv: "275c9b20dfb03e9cf72633d0d49e49dd"
server_key_version: "encryptionkey@0"
I20260806 19:41:31.927402 1078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/wal/instance:
uuid: "7745fb9c574442bf8450a52661be4c35"
format_stamp: "Formatted at 2026-08-06 19:41:31 on dist-test-slave-ss4n"
server_key: "5303eaa8b7901f5cf042b2728aa28f42"
server_key_iv: "275c9b20dfb03e9cf72633d0d49e49dd"
server_key_version: "encryptionkey@0"
I20260806 19:41:31.934190 1078 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 19:41:31.939954 1093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:31.941783 1078 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 19:41:31.942042 1078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/wal
uuid: "7745fb9c574442bf8450a52661be4c35"
format_stamp: "Formatted at 2026-08-06 19:41:31 on dist-test-slave-ss4n"
server_key: "5303eaa8b7901f5cf042b2728aa28f42"
server_key_iv: "275c9b20dfb03e9cf72633d0d49e49dd"
server_key_version: "encryptionkey@0"
I20260806 19:41:31.942350 1078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:31.962553 1078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:31.968776 1078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:31.969537 1078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:31.994287 1078 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.61:43017
I20260806 19:41:31.994330 1146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.61:43017 every 8 connection(s)
I20260806 19:41:31.995820 1078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/data/info.pb
I20260806 19:41:32.000101 1147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:32.001965 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 1078
I20260806 19:41:32.002200 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-1/wal/instance
I20260806 19:41:32.002692 10340 external_mini_cluster.cc:1428] Setting key 7929c0829dba3576da689858a088a568
I20260806 19:41:32.008106 1147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:41:32 dist-test-slave-ss4n krb5kdc[984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045291, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.61@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:41:32.037571 1150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.011194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36646 (local address 127.10.25.62:35051)
0806 19:41:32.011813 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:32.011821 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:32.012996 (+ 1175us) server_negotiation.cc:407] Connection header received
0806 19:41:32.016474 (+ 3478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:32.016480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:32.016588 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:32.016725 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:32.018695 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.019483 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.021680 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.022339 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.022552 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.027958 (+ 5406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:32.027965 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:32.027969 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:32.028024 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:32.031575 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.033711 (+ 2136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.033723 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.033729 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.034038 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.034900 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.034906 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.034909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.035323 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:32.035493 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:32.036775 (+ 1282us) server_negotiation.cc:299] Negotiation successful
0806 19:41:32.037096 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":311,"threads_started":1}
WARNING: no policy specified for kudu/127.10.25.60@KRBTEST.COM; defaulting to no policy
I20260806 19:41:32.039466 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.011771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:35051 (local address 127.0.0.1:36646)
0806 19:41:32.012790 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.012850 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:32.016036 (+ 3186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:32.017086 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:32.017096 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:32.017781 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:32.018432 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:32.018451 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.019676 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.019680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:32.020967 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:32.020989 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.022768 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.022782 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:32.023133 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.024417 (+ 1284us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:32.024457 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:32.027570 (+ 3113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:32.032008 (+ 4438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:32.032026 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:32.032059 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:32.033125 (+ 1066us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:32.034240 (+ 1115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:32.034250 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:32.034255 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:32.034679 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:32.035528 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:32.035539 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:32.036117 (+ 578us) client_negotiation.cc:769] Sending connection context
0806 19:41:32.036507 (+ 390us) client_negotiation.cc:240] Negotiation successful
0806 19:41:32.037154 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":387,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.60@KRBTEST.COM" created.
I20260806 19:41:32.042738 1147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:41:32 dist-test-slave-ss4n krb5kdc[984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045291, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.61@KRBTEST.COM for kudu/127.10.25.61@KRBTEST.COM
I20260806 19:41:32.062107 1070 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } attempt: 5
I20260806 19:41:32.063740 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.044336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:43017 (local address 127.0.0.1:42604)
0806 19:41:32.044569 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.044595 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:32.044764 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:32.046254 (+ 1490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:32.046260 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:32.046861 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:32.047025 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:32.047035 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.048775 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.048781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:32.049836 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:32.049851 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.050813 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.050819 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:32.051056 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.051774 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:32.051794 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:32.054839 (+ 3045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:32.059425 (+ 4586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:32.059434 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:32.059441 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:32.060068 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:32.060835 (+ 767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:32.060843 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:32.060849 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:32.061103 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:32.062323 (+ 1220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:32.062330 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:32.062595 (+ 265us) client_negotiation.cc:769] Sending connection context
0806 19:41:32.062793 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 19:41:32.063183 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":5}
I20260806 19:41:32.064515 1153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.044737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42604 (local address 127.10.25.61:43017)
0806 19:41:32.045690 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:32.045697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:32.045730 (+ 33us) server_negotiation.cc:407] Connection header received
0806 19:41:32.045787 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:32.045793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:32.046013 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:32.046255 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:32.047259 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.048597 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.049991 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.050639 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.050800 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.055211 (+ 4411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:32.055242 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:32.055263 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:32.055305 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:32.059200 (+ 3895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.060354 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.060366 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.060369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.060638 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.061302 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.061306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.061309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.062064 (+ 755us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:32.062514 (+ 450us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:32.063139 (+ 625us) server_negotiation.cc:299] Negotiation successful
0806 19:41:32.063530 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":567,"threads_started":1}
I20260806 19:41:32.069305 1147 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:41:32.071066 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.070212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:51928)
0806 19:41:32.070562 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.070781 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
W20260806 19:41:32.071519 1096 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
I20260806 19:41:32.074237 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.073681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:51942)
0806 19:41:32.073882 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.074001 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 19:41:32.074563 1147 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:38053: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Aug 06 19:41:32 dist-test-slave-ss4n krb5kdc[984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045291, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for kudu/127.10.25.61@KRBTEST.COM
I20260806 19:41:32.083699 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.063175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:43017 (local address 127.0.0.1:42606)
0806 19:41:32.063693 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.063718 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:32.063862 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:32.065801 (+ 1939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:32.065806 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:32.066445 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:32.066702 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:32.066713 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.068678 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.068692 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:32.070052 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:32.070185 (+ 133us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.071487 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.071494 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:32.071809 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.072625 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:32.072669 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:32.075810 (+ 3141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:32.080166 (+ 4356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:32.080172 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:32.080175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:32.080764 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:32.081458 (+ 694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:32.081463 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:32.081469 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:32.081707 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:32.082449 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:32.082455 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:32.082650 (+ 195us) client_negotiation.cc:769] Sending connection context
0806 19:41:32.082846 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 19:41:32.083246 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356}
I20260806 19:41:32.083778 1156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.064118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42606 (local address 127.10.25.61:43017)
0806 19:41:32.065254 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:32.065262 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:32.065297 (+ 35us) server_negotiation.cc:407] Connection header received
0806 19:41:32.065363 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:32.065371 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:32.065469 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:32.065783 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:32.067014 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.068413 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.070458 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.071281 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.071462 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.076105 (+ 4643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:32.076111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:32.076116 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:32.076162 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:32.079838 (+ 3676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.081069 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.081077 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.081081 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.081283 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.081887 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.081894 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.081897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.082295 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:32.082491 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:32.082967 (+ 476us) server_negotiation.cc:299] Negotiation successful
0806 19:41:32.083246 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":941,"thread_start_us":693,"threads_started":1}
I20260806 19:41:32.085907 1070 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:41:32.087517 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.086770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:51946)
0806 19:41:32.087056 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.087172 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260806 19:41:32.088994 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.088516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:51958)
0806 19:41:32.088732 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.088835 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260806 19:41:32.089335 1070 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:38053: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.60.keytab.
Entry for principal kudu/127.10.25.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.60.keytab.
I20260806 19:41:32.090978 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.60:38053
--webserver_interface=127.10.25.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.60.keytab
--principal=kudu/127.10.25.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:40487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:35051,127.10.25.61:43017,127.10.25.60:38053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:41:32.124006 1147 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } attempt: 1
I20260806 19:41:32.125365 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.124839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:51964)
0806 19:41:32.125060 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.125185 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 19:41:32.126492 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.126063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:51974)
0806 19:41:32.126267 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.126372 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 19:41:32.126729 1147 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:38053: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
I20260806 19:41:32.140769 1070 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } attempt: 1
I20260806 19:41:32.142606 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.142013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:51978)
0806 19:41:32.142270 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.142388 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260806 19:41:32.143908 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.143495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:51988)
0806 19:41:32.143693 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.143770 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260806 19:41:32.144223 1070 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:38053: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
I20260806 19:41:32.195052 1147 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } attempt: 2
I20260806 19:41:32.196400 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.195826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:52000)
0806 19:41:32.196121 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.196230 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260806 19:41:32.198077 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.197309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:52016)
0806 19:41:32.197721 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.197870 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260806 19:41:32.198525 1147 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:38053: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
I20260806 19:41:32.212548 1070 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } attempt: 2
I20260806 19:41:32.213961 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.213423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:52022)
0806 19:41:32.213667 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.213784 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260806 19:41:32.215006 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.214660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:52024)
0806 19:41:32.214823 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.214890 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 19:41:32.215238 1070 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:38053: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
I20260806 19:41:32.289896 1147 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } attempt: 3
I20260806 19:41:32.291680 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.291038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:52030)
0806 19:41:32.291369 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.291472 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260806 19:41:32.292914 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.292450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:52040)
0806 19:41:32.292707 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.292792 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260806 19:41:32.293148 1147 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:38053: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
I20260806 19:41:32.321467 1070 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } attempt: 3
I20260806 19:41:32.322907 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.322346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:52054)
0806 19:41:32.322591 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.322712 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 19:41:32.324208 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.323679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:52058)
0806 19:41:32.323925 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.324059 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260806 19:41:32.324491 1070 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:38053: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
I20260806 19:41:32.436461 1147 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } attempt: 4
I20260806 19:41:32.438290 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.437739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:52072)
0806 19:41:32.437999 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.438108 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 19:41:32.439577 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.439205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:52086)
0806 19:41:32.439365 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.439438 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260806 19:41:32.439908 1147 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:38053: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
W20260806 19:41:32.499545 1158 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 19:41:32.499962 1158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:32.500108 1158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:32.500257 1158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 19:41:32.501721 1070 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } attempt: 4
I20260806 19:41:32.503142 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.502552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:52094)
0806 19:41:32.502810 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.502930 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 19:41:32.504453 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.503982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:52104)
0806 19:41:32.504158 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.504267 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 19:41:32.504738 1070 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:38053: Network error: Client connection negotiation failed: client connection to 127.10.25.60:38053: connect: Connection refused (error 111)
W20260806 19:41:32.510568 1158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:32.510710 1158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:32.510812 1158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:32.510919 1158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:32.511014 1158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:32.524972 1158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:40487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:35051,127.10.25.61:43017,127.10.25.60:38053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.60.keytab
--principal=kudu/127.10.25.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.60:38053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.25.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:32.526752 1158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:32.528337 1158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:32.543594 1163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:32.543900 1164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:32.544764 1166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:32.547552 1158 server_base.cc:1034] running on GCE node
I20260806 19:41:32.548530 1158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:32.550312 1158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:32.551647 1158 hybrid_clock.cc:648] HybridClock initialized: now 1786045292551548 us; error 74 us; skew 500 ppm
I20260806 19:41:32.552078 1158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:32 dist-test-slave-ss4n krb5kdc[984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045292, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:32.555802 1158 init.cc:377] Logged in from keytab as kudu/127.10.25.60@KRBTEST.COM (short username kudu)
I20260806 19:41:32.557987 1158 webserver.cc:460] Webserver started at http://127.10.25.60:41581/ using document root <none> and password file <none>
I20260806 19:41:32.558827 1158 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:32.558933 1158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:32.559280 1158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:32.562788 1158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/data/instance:
uuid: "410f9cb461ec4d598766730582d894a2"
format_stamp: "Formatted at 2026-08-06 19:41:32 on dist-test-slave-ss4n"
server_key: "953f558718acf967b14f1d55dd751e49"
server_key_iv: "8c3584ba37848da2ecb74fda8279a191"
server_key_version: "encryptionkey@0"
I20260806 19:41:32.563701 1158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/wal/instance:
uuid: "410f9cb461ec4d598766730582d894a2"
format_stamp: "Formatted at 2026-08-06 19:41:32 on dist-test-slave-ss4n"
server_key: "953f558718acf967b14f1d55dd751e49"
server_key_iv: "8c3584ba37848da2ecb74fda8279a191"
server_key_version: "encryptionkey@0"
I20260806 19:41:32.570278 1158 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 19:41:32.575261 1173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:32.577698 1158 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260806 19:41:32.577967 1158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/wal
uuid: "410f9cb461ec4d598766730582d894a2"
format_stamp: "Formatted at 2026-08-06 19:41:32 on dist-test-slave-ss4n"
server_key: "953f558718acf967b14f1d55dd751e49"
server_key_iv: "8c3584ba37848da2ecb74fda8279a191"
server_key_version: "encryptionkey@0"
I20260806 19:41:32.578243 1158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:32.607457 1158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:32.613287 1158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:32.613668 1158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:32.636394 1158 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.60:38053
I20260806 19:41:32.636655 1225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.60:38053 every 8 connection(s)
I20260806 19:41:32.637945 1158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/data/info.pb
I20260806 19:41:32.638942 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 1158
I20260806 19:41:32.639192 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786044861763400-10340-0/minicluster-data/master-2/wal/instance
I20260806 19:41:32.639694 10340 external_mini_cluster.cc:1428] Setting key bf157fad3286d34d9b65377ff75f3463
I20260806 19:41:32.642887 1226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:32.651958 1226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:41:32 dist-test-slave-ss4n krb5kdc[984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.60@KRBTEST.COM
Aug 06 19:41:32 dist-test-slave-ss4n krb5kdc[984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045292, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.60@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:41:32.666705 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.642012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52112 (local address 127.10.25.60:38053)
0806 19:41:32.642874 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:32.642887 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:32.642932 (+ 45us) server_negotiation.cc:407] Connection header received
0806 19:41:32.644651 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:32.644674 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:32.645067 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:32.645563 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:32.647567 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.649046 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.651008 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.651796 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.651943 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.657773 (+ 5830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:32.657784 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:32.657798 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:32.657853 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:32.661552 (+ 3699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.662506 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.662515 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.662523 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.662739 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.663202 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.663207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.663209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.663970 (+ 761us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:32.664399 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:32.664949 (+ 550us) server_negotiation.cc:299] Negotiation successful
0806 19:41:32.665464 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":426,"threads_started":1}
I20260806 19:41:32.673239 1230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.655551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.10.25.62:35051)
0806 19:41:32.656107 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:32.656114 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:32.656683 (+ 569us) server_negotiation.cc:407] Connection header received
0806 19:41:32.656940 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:32.656950 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:32.657018 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:32.657152 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:32.658782 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.659537 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.661029 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.661474 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.661692 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.665722 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:32.665729 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:32.665733 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:32.665788 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:32.669252 (+ 3464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.670211 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.670216 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.670219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.670385 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.670853 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.670859 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.670862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.671293 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:32.671477 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:32.672700 (+ 1223us) server_negotiation.cc:299] Negotiation successful
0806 19:41:32.672909 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":281,"threads_started":1}
I20260806 19:41:32.673763 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.655621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:35051 (local address 127.0.0.1:36654)
0806 19:41:32.656509 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.656573 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:32.656784 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:32.657226 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:32.657233 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:32.658220 (+ 987us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:32.658622 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:32.658634 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.659702 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.659708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:32.660869 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:32.660882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.661747 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.661754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:32.661945 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.662621 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:32.662640 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:32.665347 (+ 2707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:32.669416 (+ 4069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:32.669431 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:32.669441 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:32.670074 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:32.670571 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:32.670575 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:32.670577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:32.670736 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:32.671718 (+ 982us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:32.671730 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:32.672128 (+ 398us) client_negotiation.cc:769] Sending connection context
0806 19:41:32.672571 (+ 443us) client_negotiation.cc:240] Negotiation successful
0806 19:41:32.673143 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":438,"threads_started":1}
I20260806 19:41:32.675810 1226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:41:32 dist-test-slave-ss4n krb5kdc[984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
Aug 06 19:41:32 dist-test-slave-ss4n krb5kdc[984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045292, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.60@KRBTEST.COM for kudu/127.10.25.61@KRBTEST.COM
I20260806 19:41:32.689226 1232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.670608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36660 (local address 127.10.25.62:35051)
0806 19:41:32.671352 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:32.671359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:32.671383 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:41:32.671445 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:32.671450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:32.671527 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:32.671608 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:32.673982 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.675113 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.676869 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.677539 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.677683 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.682395 (+ 4712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:32.682404 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:32.682409 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:32.682465 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:32.685823 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.686715 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.686724 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.686727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.686887 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.687454 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.687461 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.687466 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.687882 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:32.688208 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:32.688503 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 19:41:32.688752 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":375,"threads_started":1}
I20260806 19:41:32.694372 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.676646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:43017 (local address 127.0.0.1:42620)
0806 19:41:32.676979 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.676999 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:32.677121 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:32.678049 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:32.678055 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:32.678587 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:32.678895 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:32.678913 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.680093 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.680099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:32.681059 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:32.681078 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.682186 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.682191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:32.682448 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.683229 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:32.683263 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:32.686130 (+ 2867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:32.690311 (+ 4181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:32.690317 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:32.690321 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:32.691140 (+ 819us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:32.691832 (+ 692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:32.691840 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:32.691846 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:32.692149 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:32.692956 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:32.692964 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:32.693231 (+ 267us) client_negotiation.cc:769] Sending connection context
0806 19:41:32.693457 (+ 226us) client_negotiation.cc:240] Negotiation successful
0806 19:41:32.693916 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201}
I20260806 19:41:32.694451 1233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.676902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42620 (local address 127.10.25.61:43017)
0806 19:41:32.677660 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:32.677665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:32.677684 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:41:32.677760 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:32.677765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:32.677839 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:32.678001 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:32.679159 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.679927 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.681244 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.682001 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.682153 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.686422 (+ 4269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:32.686428 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:32.686432 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:32.686491 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:32.690115 (+ 3624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.691421 (+ 1306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.691427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.691430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.691649 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.692295 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.692301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.692305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.692728 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:32.692871 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:32.693773 (+ 902us) server_negotiation.cc:299] Negotiation successful
0806 19:41:32.694055 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":375,"threads_started":1}
I20260806 19:41:32.696266 1226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:41:32 dist-test-slave-ss4n krb5kdc[984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.61@KRBTEST.COM
Aug 06 19:41:32 dist-test-slave-ss4n krb5kdc[984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045292, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.60@KRBTEST.COM for kudu/127.10.25.60@KRBTEST.COM
I20260806 19:41:32.713402 1234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.691588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42624 (local address 127.10.25.61:43017)
0806 19:41:32.692292 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:32.692299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:32.692315 (+ 16us) server_negotiation.cc:407] Connection header received
0806 19:41:32.692362 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:32.692367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:32.692481 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:32.692606 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:32.694830 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.695944 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.697854 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.698488 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.698600 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.705342 (+ 6742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:32.705349 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:32.705353 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:32.705395 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:32.709206 (+ 3811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.710326 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.710333 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.710336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.710514 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.711082 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.711227 (+ 145us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.711275 (+ 48us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.711869 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:32.712107 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:32.712611 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 19:41:32.712924 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":544,"thread_start_us":388,"threads_started":1}
I20260806 19:41:32.713879 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.697410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52124 (local address 127.10.25.60:38053)
0806 19:41:32.697633 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:32.697637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:32.697656 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:41:32.697866 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:32.697871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:32.697954 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:32.698068 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:32.699677 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.700861 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.702321 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.702800 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.702965 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.706805 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:32.706810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:32.706813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:32.706844 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:32.709985 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.710945 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.710951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.710954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.711120 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.711864 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.711868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.711871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.712373 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:32.712526 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:32.713049 (+ 523us) server_negotiation.cc:299] Negotiation successful
0806 19:41:32.713415 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":106,"server-negotiator.queue_time_us":116}
I20260806 19:41:32.714171 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.697229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:52124)
0806 19:41:32.697466 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.697516 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:32.697688 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:32.698294 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:32.698300 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:32.699100 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:32.699393 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:32.699411 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.701049 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.701057 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:32.702140 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:32.702154 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.703004 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.703011 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:32.703261 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.704014 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:32.704052 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:32.706592 (+ 2540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:32.710177 (+ 3585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:32.710182 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:32.710184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:32.710780 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:32.711461 (+ 681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:32.711467 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:32.711471 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:32.711675 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:32.712522 (+ 847us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:32.712527 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:32.712717 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 19:41:32.712877 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 19:41:32.713224 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 19:41:32.719606 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:41:32.720626 1226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2: Bootstrap starting.
I20260806 19:41:32.724716 1226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:32.725978 1226 log.cc:824] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:32.723584 1043 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "663b4999aec244bf86ee3c94e1c8d08a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36660
I20260806 19:41:32.730410 1226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2: No bootstrap required, opened a new log
I20260806 19:41:32.739193 1147 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } attempt: 5
I20260806 19:41:32.743846 1226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } }
I20260806 19:41:32.744540 1226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 410f9cb461ec4d598766730582d894a2, State: Initialized, Role: FOLLOWER
I20260806 19:41:32.745674 1226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } }
I20260806 19:41:32.747103 1236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } } }
I20260806 19:41:32.747577 1236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:41:32.748642 1226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2 [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 19:41:32 dist-test-slave-ss4n krb5kdc[984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045291, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.61@KRBTEST.COM for kudu/127.10.25.60@KRBTEST.COM
I20260806 19:41:32.759148 1151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.740097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:52136)
0806 19:41:32.740324 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.740346 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:32.740479 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:32.741450 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:32.741457 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:32.742014 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:32.742292 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:32.742310 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.743974 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.743980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:32.745259 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:32.745277 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.746279 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.746285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:32.746494 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.747327 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:32.747357 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:32.750227 (+ 2870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:32.755671 (+ 5444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:32.755678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:32.755682 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:32.756271 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:32.756827 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:32.756832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:32.756835 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:32.757012 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:32.757839 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:32.757846 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:32.758045 (+ 199us) client_negotiation.cc:769] Sending connection context
0806 19:41:32.758223 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 19:41:32.758628 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 19:41:32.760707 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.740747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52136 (local address 127.10.25.60:38053)
0806 19:41:32.741049 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:32.741066 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:32.741102 (+ 36us) server_negotiation.cc:407] Connection header received
0806 19:41:32.741170 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:32.741178 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:32.741258 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:32.742489 (+ 1231us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:32.742658 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.743796 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.745446 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.746123 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.746247 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.750452 (+ 4205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:32.750457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:32.750463 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:32.750501 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:32.755440 (+ 4939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.756463 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.756472 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.756477 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.756683 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.757157 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.757162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.757166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.757651 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:32.759697 (+ 2046us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:32.759869 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 19:41:32.760180 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260806 19:41:32.761384 1247 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:41:32.761624 1247 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:41:32.767846 1147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35: Bootstrap starting.
I20260806 19:41:32.758579 1158 master_runner.cc:186] Error getting master registration for 127.10.25.62:35051: OK, instance_id { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" instance_seqno: 1786045291334211 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 19:41:32.768872 1158 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 19:41:32.773344 1147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:32.775068 1147 log.cc:824] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:32.778935 1147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35: No bootstrap required, opened a new log
I20260806 19:41:32.781994 1070 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } attempt: 5
I20260806 19:41:32.791500 1147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } }
I20260806 19:41:32.792009 1147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7745fb9c574442bf8450a52661be4c35, State: Initialized, Role: FOLLOWER
Aug 06 19:41:32 dist-test-slave-ss4n krb5kdc[984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045291, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for kudu/127.10.25.60@KRBTEST.COM
I20260806 19:41:32.792656 1147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } }
I20260806 19:41:32.794174 1250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } } }
I20260806 19:41:32.795125 1250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:41:32.795491 1147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:32.804154 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.782954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:38053 (local address 127.0.0.1:52144)
0806 19:41:32.783200 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:32.783223 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:32.783382 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:32.783969 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:32.783974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:32.784656 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:32.784893 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:32.784907 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.786542 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.786549 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:32.788073 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:32.788093 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.789375 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.789384 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:32.789645 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.790413 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:32.790440 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:32.793634 (+ 3194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:32.799849 (+ 6215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:32.799859 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:32.799866 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:32.800683 (+ 817us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:32.801339 (+ 656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:32.801347 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:32.801353 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:32.801647 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:32.802616 (+ 969us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:32.802624 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:32.802872 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 19:41:32.803067 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 19:41:32.803500 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 19:41:32.805730 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.783358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52144 (local address 127.10.25.60:38053)
0806 19:41:32.783556 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:32.783563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:32.783587 (+ 24us) server_negotiation.cc:407] Connection header received
0806 19:41:32.783656 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:32.783662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:32.783814 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:32.783925 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:32.785076 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.786363 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.788275 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.788984 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.789124 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.793926 (+ 4802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:32.793934 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:32.793938 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:32.793996 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:32.799483 (+ 5487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.800907 (+ 1424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.800916 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.800922 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.801159 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.801818 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.801823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.801826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.802305 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:32.802515 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:32.804745 (+ 2230us) server_negotiation.cc:299] Negotiation successful
0806 19:41:32.805122 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 19:41:32.813257 1070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a: Bootstrap starting.
I20260806 19:41:32.822484 1070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:32.826422 1070 log.cc:824] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a: Log is configured to *not* fsync() on all Append() calls
W20260806 19:41:32.830838 1261 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 19:41:32.831040 1261 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:41:32.816004 1078 master_runner.cc:186] Error getting master registration for 127.10.25.62:35051: OK, instance_id { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" instance_seqno: 1786045291334211 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 19:41:32.832731 1078 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 19:41:32.834225 1070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a: No bootstrap required, opened a new log
I20260806 19:41:32.842744 1070 raft_consensus.cc:348] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } }
I20260806 19:41:32.843231 1070 raft_consensus.cc:729] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 663b4999aec244bf86ee3c94e1c8d08a, State: Initialized, Role: FOLLOWER
I20260806 19:41:32.843830 1070 consensus_queue.cc:260] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } }
I20260806 19:41:32.845528 1264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } } }
I20260806 19:41:32.846387 1264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:41:32.846812 1070 sys_catalog.cc:564] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:32.847112 1043 raft_consensus.cc:482] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 19:41:32.847558 1043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:32.851606 1043 raft_consensus.cc:504] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } }
I20260806 19:41:32.854439 1120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "663b4999aec244bf86ee3c94e1c8d08a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7745fb9c574442bf8450a52661be4c35"
I20260806 19:41:32.855037 1120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:32.859246 1120 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663b4999aec244bf86ee3c94e1c8d08a in term 1.
I20260806 19:41:32.860131 1016 leader_election.cc:302] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 663b4999aec244bf86ee3c94e1c8d08a, 7745fb9c574442bf8450a52661be4c35; no voters:
I20260806 19:41:32.862047 1200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "663b4999aec244bf86ee3c94e1c8d08a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "410f9cb461ec4d598766730582d894a2"
I20260806 19:41:32.862622 1200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2 [term 0 FOLLOWER]: Advancing to term 1
W20260806 19:41:32.866211 1179 tablet.cc:2369] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:41:32.866751 1200 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663b4999aec244bf86ee3c94e1c8d08a in term 1.
I20260806 19:41:32.867687 1043 leader_election.cc:288] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [CANDIDATE]: Term 1 election: Requested vote from peers 7745fb9c574442bf8450a52661be4c35 (127.10.25.61:43017), 410f9cb461ec4d598766730582d894a2 (127.10.25.60:38053)
I20260806 19:41:32.875190 1264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:32.883085 1264 raft_consensus.cc:686] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [term 1 LEADER]: Becoming Leader. State: Replica: 663b4999aec244bf86ee3c94e1c8d08a, State: Running, Role: LEADER
I20260806 19:41:32.884955 1264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } }
I20260806 19:41:32.914324 1269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 663b4999aec244bf86ee3c94e1c8d08a. Latest consensus state: current_term: 1 leader_uuid: "663b4999aec244bf86ee3c94e1c8d08a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } } }
I20260806 19:41:32.915316 1269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:32.922657 1279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 19:41:32.946458 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.903527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52146 (local address 127.10.25.60:38053)
0806 19:41:32.903766 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:32.903772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:32.910288 (+ 6516us) server_negotiation.cc:407] Connection header received
0806 19:41:32.910610 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:32.910618 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:32.910742 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:32.910835 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:32.925667 (+ 14832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.926889 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.930053 (+ 3164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.931283 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.931510 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.937117 (+ 5607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:32.937127 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:32.937133 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:32.937203 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:32.941766 (+ 4563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.942922 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.942932 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.942938 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.943134 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.943670 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.943676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.943680 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.944137 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:32.944437 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:32.945650 (+ 1213us) server_negotiation.cc:299] Negotiation successful
0806 19:41:32.945970 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 19:41:32.963528 1234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.894498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42626 (local address 127.10.25.61:43017)
0806 19:41:32.894732 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:32.894743 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:32.902844 (+ 8101us) server_negotiation.cc:407] Connection header received
0806 19:41:32.905243 (+ 2399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:32.905251 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:32.905355 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:32.905439 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:32.917305 (+ 11866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.918648 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.925893 (+ 7245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.926779 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.926967 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.946322 (+ 19355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:32.946330 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:32.946334 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:32.946387 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:32.951240 (+ 4853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.954325 (+ 3085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.954333 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.954338 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.954581 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.955120 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.955127 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.955131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.955550 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:32.955874 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:32.962748 (+ 6874us) server_negotiation.cc:299] Negotiation successful
0806 19:41:32.963060 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 19:41:32.973289 1099 tablet.cc:2369] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:41:32.973968 1279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:32.977916 1232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:32.895491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36676 (local address 127.10.25.62:35051)
0806 19:41:32.895762 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:32.895771 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:32.904906 (+ 9135us) server_negotiation.cc:407] Connection header received
0806 19:41:32.909301 (+ 4395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:32.909310 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:32.909461 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:32.909621 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:32.925663 (+ 16042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.938334 (+ 12671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.940783 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:32.941686 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:32.941867 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:32.956590 (+ 14723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:32.956600 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:32.956607 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:32.956659 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:32.961239 (+ 4580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.967285 (+ 6046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.967293 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.967298 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.967540 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:32.974825 (+ 7285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:32.974833 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:32.974837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:32.975329 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:32.975668 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:32.977140 (+ 1472us) server_negotiation.cc:299] Negotiation successful
0806 19:41:32.977434 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 19:41:32.981225 1288 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 19:41:33.046173 1200 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2 [term 1 FOLLOWER]: Refusing update from remote peer 663b4999aec244bf86ee3c94e1c8d08a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:41:33.047912 1269 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [LEADER]: Connected to new peer: Peer: permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:41:33.062053 1120 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35 [term 1 FOLLOWER]: Refusing update from remote peer 663b4999aec244bf86ee3c94e1c8d08a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:41:33.063886 1269 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 19:41:33.071363 1020 tablet.cc:2369] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:41:33.104102 1250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 663b4999aec244bf86ee3c94e1c8d08a. Latest consensus state: current_term: 1 leader_uuid: "663b4999aec244bf86ee3c94e1c8d08a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } } }
I20260806 19:41:33.104480 1250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:41:33.109194 1289 mvcc.cc:205] Tried to move back new op lower bound from 7315641520301744128 to 7315641519678435328. Current Snapshot: MvccSnapshot[applied={T|T < 7315641520301744128}]
I20260806 19:41:33.110991 1236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 663b4999aec244bf86ee3c94e1c8d08a. Latest consensus state: current_term: 1 leader_uuid: "663b4999aec244bf86ee3c94e1c8d08a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } } }
I20260806 19:41:33.111353 1236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:41:33.117327 1290 mvcc.cc:205] Tried to move back new op lower bound from 7315641520301744128 to 7315641519678435328. Current Snapshot: MvccSnapshot[applied={T|T < 7315641520301744128}]
I20260806 19:41:33.128064 1236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "663b4999aec244bf86ee3c94e1c8d08a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } } }
I20260806 19:41:33.128340 1236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:41:33.129154 1250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "663b4999aec244bf86ee3c94e1c8d08a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } } }
I20260806 19:41:33.129426 1250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:41:33.140944 1264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "663b4999aec244bf86ee3c94e1c8d08a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } } }
I20260806 19:41:33.141705 1264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:33.142110 1264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "663b4999aec244bf86ee3c94e1c8d08a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663b4999aec244bf86ee3c94e1c8d08a" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 35051 } } peers { permanent_uuid: "7745fb9c574442bf8450a52661be4c35" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 43017 } } peers { permanent_uuid: "410f9cb461ec4d598766730582d894a2" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 38053 } } }
I20260806 19:41:33.142396 1264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:33.144526 1279 catalog_manager.cc:1347] Generated new cluster ID: 578e31201b4f411286e246e16ea43d1f
I20260806 19:41:33.144646 1279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:33.185058 1279 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:33.187206 1279 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:33.225203 1279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 663b4999aec244bf86ee3c94e1c8d08a: Generated new TSK 0
I20260806 19:41:33.226498 1279 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 19:41:33.386886 1234 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:33.364840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42642 (local address 127.10.25.61:43017)
0806 19:41:33.365033 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:33.365040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:33.383373 (+ 18333us) server_negotiation.cc:407] Connection header received
0806 19:41:33.383718 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:33.383725 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:33.383822 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:33.383902 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:33.386615 (+ 2713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42642: BlockingRecv error: recv got EOF from 127.0.0.1:42642 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 19:41:33.386997 1232 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:33.363943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36690 (local address 127.10.25.62:35051)
0806 19:41:33.364181 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:33.364188 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:33.382397 (+ 18209us) server_negotiation.cc:407] Connection header received
0806 19:41:33.382916 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:33.382923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:33.383073 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:33.383205 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:33.386690 (+ 3485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36690: BlockingRecv error: recv got EOF from 127.0.0.1:36690 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
W20260806 19:41:33.397406 1229 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:33.384648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52156 (local address 127.10.25.60:38053)
0806 19:41:33.384907 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:33.384917 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:33.387869 (+ 2952us) server_negotiation.cc:407] Connection header received
0806 19:41:33.395844 (+ 7975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:33.395850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:33.395944 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:33.396027 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:33.397152 (+ 1125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52156: BlockingRecv error: recv got EOF from 127.0.0.1:52156 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260806 19:41:33.444512 1234 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:33.435748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42658 (local address 127.10.25.61:43017)
0806 19:41:33.435993 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:33.436003 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:33.442305 (+ 6302us) server_negotiation.cc:407] Connection header received
0806 19:41:33.442769 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:33.442775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:33.442968 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:33.443064 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:33.444246 (+ 1182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42658: BlockingRecv error: recv got EOF from 127.0.0.1:42658 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 19:41:33.495868 1234 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:33.482067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42670 (local address 127.10.25.61:43017)
0806 19:41:33.482382 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:33.482388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:33.494248 (+ 11860us) server_negotiation.cc:407] Connection header received
0806 19:41:33.494491 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:33.494499 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:33.494590 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:33.494722 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:33.495550 (+ 828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42670: BlockingRecv error: recv got EOF from 127.0.0.1:42670 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
W20260806 19:41:33.505261 1234 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:33.502696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42674 (local address 127.10.25.61:43017)
0806 19:41:33.502976 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:33.502986 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:33.503573 (+ 587us) server_negotiation.cc:407] Connection header received
0806 19:41:33.503830 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:33.503837 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:33.503933 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:33.504057 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:33.504967 (+ 910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42674: BlockingRecv error: recv got EOF from 127.0.0.1:42674 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260806 19:41:33.515283 1234 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:33.512895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42684 (local address 127.10.25.61:43017)
0806 19:41:33.513187 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:33.513196 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:33.513785 (+ 589us) server_negotiation.cc:407] Connection header received
0806 19:41:33.514017 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:33.514023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:33.514112 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:33.514229 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:33.515010 (+ 781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42684: BlockingRecv error: recv got EOF from 127.0.0.1:42684 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
W20260806 19:41:33.536170 1234 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:33.527854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42694 (local address 127.10.25.61:43017)
0806 19:41:33.528146 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:33.528158 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:33.534333 (+ 6175us) server_negotiation.cc:407] Connection header received
0806 19:41:33.534658 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:33.534665 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:33.534769 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:33.534911 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:33.535854 (+ 943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42694: BlockingRecv error: recv got EOF from 127.0.0.1:42694 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260806 19:41:33.559183 1234 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:33.556134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42706 (local address 127.10.25.61:43017)
0806 19:41:33.556420 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:33.556430 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:33.557077 (+ 647us) server_negotiation.cc:407] Connection header received
0806 19:41:33.557348 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:33.557353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:33.557434 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:33.557605 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:33.558782 (+ 1177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42706: BlockingRecv error: recv got EOF from 127.0.0.1:42706 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 19:41:33.599176 1234 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:33.596207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42714 (local address 127.10.25.61:43017)
0806 19:41:33.596532 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:33.596541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:33.597123 (+ 582us) server_negotiation.cc:407] Connection header received
0806 19:41:33.597420 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:33.597429 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:33.597600 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:33.597785 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:33.598789 (+ 1004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42714: BlockingRecv error: recv got EOF from 127.0.0.1:42714 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
W20260806 19:41:33.669827 1234 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:33.667109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42724 (local address 127.10.25.61:43017)
0806 19:41:33.667383 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:33.667390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:33.668010 (+ 620us) server_negotiation.cc:407] Connection header received
0806 19:41:33.668250 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:33.668255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:33.668377 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:33.668501 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:33.669537 (+ 1036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42724: BlockingRecv error: recv got EOF from 127.0.0.1:42724 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 19:41:33.763849 1247 catalog_manager.cc:1259] Loaded cluster ID: 578e31201b4f411286e246e16ea43d1f
I20260806 19:41:33.764034 1247 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2: loading cluster ID for follower catalog manager: success
I20260806 19:41:33.766844 1247 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2: acquiring CA information for follower catalog manager: success
I20260806 19:41:33.768883 1247 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 410f9cb461ec4d598766730582d894a2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 19:41:33.804222 1234 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:33.802311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42736 (local address 127.10.25.61:43017)
0806 19:41:33.802561 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:33.802567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:33.802716 (+ 149us) server_negotiation.cc:407] Connection header received
0806 19:41:33.802844 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:33.802849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:33.802955 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:33.803094 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:33.803952 (+ 858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42736: BlockingRecv error: recv got EOF from 127.0.0.1:42736 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 19:41:33.833300 1261 catalog_manager.cc:1259] Loaded cluster ID: 578e31201b4f411286e246e16ea43d1f
I20260806 19:41:33.833426 1261 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35: loading cluster ID for follower catalog manager: success
I20260806 19:41:33.836303 1261 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35: acquiring CA information for follower catalog manager: success
I20260806 19:41:33.838893 1261 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7745fb9c574442bf8450a52661be4c35: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 19:41:34.071915 1234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:34.064522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42738 (local address 127.10.25.61:43017)
0806 19:41:34.064715 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:34.064720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:34.064891 (+ 171us) server_negotiation.cc:407] Connection header received
0806 19:41:34.065074 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:34.065079 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:34.065242 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:34.065355 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:41:34.066249 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:34.067113 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:34.069080 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:34.069593 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:34.069717 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:34.070642 (+ 925us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:41:34.071222 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:41:34.071348 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:34.071555 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 19:41:34.071659 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 19:41:34.089237 1363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:34.080030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52160 (local address 127.10.25.60:38053)
0806 19:41:34.080709 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:34.080715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:34.081327 (+ 612us) server_negotiation.cc:407] Connection header received
0806 19:41:34.081840 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:34.081847 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:34.082037 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:34.082250 (+ 213us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:41:34.083025 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:34.084261 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:34.086011 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:34.086493 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:34.086643 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:34.087365 (+ 722us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:41:34.088250 (+ 885us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:41:34.088441 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:34.088687 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 19:41:34.088845 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":460,"threads_started":1}
I20260806 19:41:34.109553 1234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:34.099136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42752 (local address 127.10.25.61:43017)
0806 19:41:34.099420 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:34.099430 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:34.100927 (+ 1497us) server_negotiation.cc:407] Connection header received
0806 19:41:34.101315 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:34.101322 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:34.101449 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:34.101586 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:41:34.102485 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:34.103613 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:34.106738 (+ 3125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:34.107551 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:34.107725 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:34.108558 (+ 833us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:41:34.108724 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:41:34.108838 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:34.109066 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 19:41:34.109179 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 19:41:34.110460 1363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:34.101254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52176 (local address 127.10.25.60:38053)
0806 19:41:34.101858 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:34.101865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:34.101939 (+ 74us) server_negotiation.cc:407] Connection header received
0806 19:41:34.102161 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:34.102188 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:34.102258 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:34.102339 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:41:34.102962 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:34.104261 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:34.107228 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:34.107779 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:34.107992 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:34.108970 (+ 978us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:41:34.109130 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:41:34.109219 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:34.110036 (+ 817us) server_negotiation.cc:299] Negotiation successful
0806 19:41:34.110151 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454}
I20260806 19:41:34.110690 1369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:34.099212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36706 (local address 127.10.25.62:35051)
0806 19:41:34.100055 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:34.100064 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:34.100986 (+ 922us) server_negotiation.cc:407] Connection header received
0806 19:41:34.101505 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:34.101517 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:34.101766 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:34.101970 (+ 204us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:41:34.103227 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:34.104522 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:34.107742 (+ 3220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:34.108357 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:34.108535 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:34.109301 (+ 766us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:41:34.109835 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:41:34.109920 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:34.110179 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 19:41:34.110281 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":381,"threads_started":1}
I20260806 19:41:34.118257 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 998
I20260806 19:41:34.119848 1065 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:34.242729 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 998
W20260806 19:41:34.266707 1094 connection.cc:535] server connection from 127.0.0.1:42606 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 19:41:34.266705 1175 connection.cc:535] server connection from 127.0.0.1:52144 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 19:41:34.267105 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 1078
I20260806 19:41:34.268483 1142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:34.382200 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 1078
I20260806 19:41:34.407975 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 1158
I20260806 19:41:34.409564 1221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:34.522229 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 1158
2026-08-06T19:41:34Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5406 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 19:41:34 dist-test-slave-ss4n krb5kdc[1378](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 19:41:34 dist-test-slave-ss4n krb5kdc[1378](info): set up 2 sockets
Aug 06 19:41:34 dist-test-slave-ss4n krb5kdc[1378](info): commencing operation
krb5kdc: starting...
W20260806 19:41:35.768998 10340 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.001s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/test-user.keytab.
Aug 06 19:41:36 dist-test-slave-ss4n krb5kdc[1378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T19:41:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T19:41:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.25.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
Entry for principal kudu/127.10.25.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab.
I20260806 19:41:36.175328 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:43163
--webserver_interface=127.10.25.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:43879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:43163,127.10.25.61:45701,127.10.25.60:42241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 19:41:36.594142 1391 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 19:41:36.594626 1391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:36.594770 1391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:36.594935 1391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:36.605248 1391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:36.605453 1391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:36.605708 1391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:36.605813 1391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:36.605959 1391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:36.617873 1391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:43879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:43163,127.10.25.61:45701,127.10.25.60:42241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.62.keytab
--principal=kudu/127.10.25.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.62:43163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.25.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:36.619370 1391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:36.621140 1391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:36.635216 1396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:36.635713 1399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:36.635879 1397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:36.636819 1391 server_base.cc:1034] running on GCE node
I20260806 19:41:36.637825 1391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:36.639796 1391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:36.641140 1391 hybrid_clock.cc:648] HybridClock initialized: now 1786045296641169 us; error 170 us; skew 500 ppm
I20260806 19:41:36.641664 1391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:36 dist-test-slave-ss4n krb5kdc[1378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:36.651551 1391 init.cc:377] Logged in from keytab as kudu/127.10.25.62@KRBTEST.COM (short username kudu)
I20260806 19:41:36.653824 1391 webserver.cc:460] Webserver started at http://127.10.25.62:39715/ using document root <none> and password file <none>
I20260806 19:41:36.654927 1391 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:36.655117 1391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:36.655608 1391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:36.659437 1391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/data/instance:
uuid: "bdb659eba3eb4e28b278c77ea3589434"
format_stamp: "Formatted at 2026-08-06 19:41:36 on dist-test-slave-ss4n"
server_key: "32e7ea183cdc03419f655510185f5c64"
server_key_iv: "a96b23007d4deac3425b449308cf8645"
server_key_version: "encryptionkey@0"
I20260806 19:41:36.660446 1391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/wal/instance:
uuid: "bdb659eba3eb4e28b278c77ea3589434"
format_stamp: "Formatted at 2026-08-06 19:41:36 on dist-test-slave-ss4n"
server_key: "32e7ea183cdc03419f655510185f5c64"
server_key_iv: "a96b23007d4deac3425b449308cf8645"
server_key_version: "encryptionkey@0"
I20260806 19:41:36.668115 1391 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260806 19:41:36.673827 1407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:36.676373 1391 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260806 19:41:36.676685 1391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/wal
uuid: "bdb659eba3eb4e28b278c77ea3589434"
format_stamp: "Formatted at 2026-08-06 19:41:36 on dist-test-slave-ss4n"
server_key: "32e7ea183cdc03419f655510185f5c64"
server_key_iv: "a96b23007d4deac3425b449308cf8645"
server_key_version: "encryptionkey@0"
I20260806 19:41:36.676982 1391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:36.728755 1391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:36.735121 1391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:36.735461 1391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:36.763290 1391 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.62:43163
I20260806 19:41:36.763304 1459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.62:43163 every 8 connection(s)
I20260806 19:41:36.765067 1391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/data/info.pb
I20260806 19:41:36.766069 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 1391
I20260806 19:41:36.766309 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-0/wal/instance
I20260806 19:41:36.767345 10340 external_mini_cluster.cc:1428] Setting key 18cdc03216f6296bb54f7f3a3275764e
I20260806 19:41:36.770517 1460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:36.781967 1460 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:41:36 dist-test-slave-ss4n krb5kdc[1378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.25.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.61@KRBTEST.COM" created.
I20260806 19:41:36.808764 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:36.786181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43163 (local address 127.0.0.1:34378)
0806 19:41:36.787246 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:36.787310 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:36.789703 (+ 2393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:36.791177 (+ 1474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:36.791184 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:36.791997 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:36.792422 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:36.792447 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:36.794264 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:36.794269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:36.795404 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:36.795417 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:36.796447 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:36.796454 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:36.796630 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:36.797406 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:36.797443 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:36.799873 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:36.804399 (+ 4526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:36.804413 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:36.804428 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:36.804870 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:36.805385 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:36.805390 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:36.805394 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:36.805634 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:36.806638 (+ 1004us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:36.806657 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:36.806913 (+ 256us) client_negotiation.cc:769] Sending connection context
0806 19:41:36.807152 (+ 239us) client_negotiation.cc:240] Negotiation successful
0806 19:41:36.807595 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":488,"threads_started":1}
I20260806 19:41:36.808831 1464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:36.786182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34378 (local address 127.10.25.62:43163)
0806 19:41:36.788205 (+ 2023us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:36.788220 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:36.788277 (+ 57us) server_negotiation.cc:407] Connection header received
0806 19:41:36.790573 (+ 2296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:36.790582 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:36.790894 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:36.791166 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:36.792704 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:36.794076 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:36.795573 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:36.796173 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:36.796402 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:36.800155 (+ 3753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:36.800164 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:36.800179 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:36.800216 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:36.804188 (+ 3972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:36.805062 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:36.805074 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:36.805080 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:36.805256 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:36.805752 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:36.805758 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:36.805763 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:36.806359 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:36.806603 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:36.807437 (+ 834us) server_negotiation.cc:299] Negotiation successful
0806 19:41:36.807759 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1773,"thread_start_us":1519,"threads_started":1}
I20260806 19:41:36.812812 1460 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:41:36.814589 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:36.813928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:45701 (local address 127.0.0.1:58458)
0806 19:41:36.814224 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:36.814393 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:45701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260806 19:41:36.814998 1408 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.25.61:45701: connect: Connection refused (error 111)
I20260806 19:41:36.817267 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:36.816681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:45701 (local address 127.0.0.1:58468)
0806 19:41:36.816907 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:36.817015 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:45701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260806 19:41:36.817577 1460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.61:45701: Network error: Client connection negotiation failed: client connection to 127.10.25.61:45701: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.61.keytab.
Entry for principal kudu/127.10.25.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.61.keytab.
I20260806 19:41:36.851671 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.61:45701
--webserver_interface=127.10.25.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.61.keytab
--principal=kudu/127.10.25.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:43879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:43163,127.10.25.61:45701,127.10.25.60:42241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:41:36.867024 1460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } attempt: 1
I20260806 19:41:36.868675 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:36.868141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:45701 (local address 127.0.0.1:58470)
0806 19:41:36.868379 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:36.868486 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:45701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 19:41:36.870381 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:36.869841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:45701 (local address 127.0.0.1:58480)
0806 19:41:36.870094 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:36.870196 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:45701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260806 19:41:36.870648 1460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.61:45701: Network error: Client connection negotiation failed: client connection to 127.10.25.61:45701: connect: Connection refused (error 111)
I20260806 19:41:36.939121 1460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } attempt: 2
I20260806 19:41:36.940972 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:36.940038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:45701 (local address 127.0.0.1:58492)
0806 19:41:36.940483 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:36.940649 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:45701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209}
I20260806 19:41:36.942603 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:36.941939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:45701 (local address 127.0.0.1:58498)
0806 19:41:36.942148 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:36.942302 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:45701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 19:41:36.942885 1460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.61:45701: Network error: Client connection negotiation failed: client connection to 127.10.25.61:45701: connect: Connection refused (error 111)
I20260806 19:41:37.034278 1460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } attempt: 3
I20260806 19:41:37.036185 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.035276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:45701 (local address 127.0.0.1:58504)
0806 19:41:37.035712 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.035884 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:45701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253}
I20260806 19:41:37.037667 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.037103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:45701 (local address 127.0.0.1:58520)
0806 19:41:37.037292 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.037394 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:45701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 19:41:37.037909 1460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.61:45701: Network error: Client connection negotiation failed: client connection to 127.10.25.61:45701: connect: Connection refused (error 111)
I20260806 19:41:37.181388 1460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } attempt: 4
I20260806 19:41:37.183339 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.182489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:45701 (local address 127.0.0.1:58524)
0806 19:41:37.182963 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.183110 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:45701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235}
I20260806 19:41:37.185277 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.184668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:45701 (local address 127.0.0.1:58538)
0806 19:41:37.184905 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.185065 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.61:45701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260806 19:41:37.185635 1460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.61:45701: Network error: Client connection negotiation failed: client connection to 127.10.25.61:45701: connect: Connection refused (error 111)
W20260806 19:41:37.289372 1467 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 19:41:37.289881 1467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:37.289981 1467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:37.290115 1467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:37.297943 1467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:37.298064 1467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:37.298168 1467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:37.298270 1467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:37.298384 1467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:37.309334 1467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:43879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:43163,127.10.25.61:45701,127.10.25.60:42241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.61.keytab
--principal=kudu/127.10.25.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.61:45701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.25.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:37.310881 1467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:37.312309 1467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:37.326190 1476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:37.326190 1474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:37.326378 1472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:37.327198 1467 server_base.cc:1034] running on GCE node
I20260806 19:41:37.328202 1467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:37.330044 1467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:37.331341 1467 hybrid_clock.cc:648] HybridClock initialized: now 1786045297331233 us; error 70 us; skew 500 ppm
I20260806 19:41:37.331844 1467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:37 dist-test-slave-ss4n krb5kdc[1378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045297, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:37.337356 1467 init.cc:377] Logged in from keytab as kudu/127.10.25.61@KRBTEST.COM (short username kudu)
I20260806 19:41:37.339897 1467 webserver.cc:460] Webserver started at http://127.10.25.61:45969/ using document root <none> and password file <none>
I20260806 19:41:37.341349 1467 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:37.341578 1467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:37.342051 1467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:37.345935 1467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/data/instance:
uuid: "f3c7b0ebc590426589c7a2d54f081de9"
format_stamp: "Formatted at 2026-08-06 19:41:37 on dist-test-slave-ss4n"
server_key: "74c0f7b3c33a3e8a72512122fa930f62"
server_key_iv: "2efb2e857fb482671f13e2c8fe376afc"
server_key_version: "encryptionkey@0"
I20260806 19:41:37.347005 1467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/wal/instance:
uuid: "f3c7b0ebc590426589c7a2d54f081de9"
format_stamp: "Formatted at 2026-08-06 19:41:37 on dist-test-slave-ss4n"
server_key: "74c0f7b3c33a3e8a72512122fa930f62"
server_key_iv: "2efb2e857fb482671f13e2c8fe376afc"
server_key_version: "encryptionkey@0"
I20260806 19:41:37.354211 1467 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260806 19:41:37.359647 1483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:37.361776 1467 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.001s
I20260806 19:41:37.362098 1467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/wal
uuid: "f3c7b0ebc590426589c7a2d54f081de9"
format_stamp: "Formatted at 2026-08-06 19:41:37 on dist-test-slave-ss4n"
server_key: "74c0f7b3c33a3e8a72512122fa930f62"
server_key_iv: "2efb2e857fb482671f13e2c8fe376afc"
server_key_version: "encryptionkey@0"
I20260806 19:41:37.362437 1467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:37.381850 1467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:37.387799 1467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:37.388160 1467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:37.415304 1467 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.61:45701
I20260806 19:41:37.415333 1535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.61:45701 every 8 connection(s)
I20260806 19:41:37.417030 1467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/data/info.pb
I20260806 19:41:37.421425 1536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:37.427345 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 1467
I20260806 19:41:37.427654 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-1/wal/instance
I20260806 19:41:37.428548 10340 external_mini_cluster.cc:1428] Setting key 5eeadd99e91014a0587b0b08d0b92548
I20260806 19:41:37.434775 1536 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:41:37 dist-test-slave-ss4n krb5kdc[1378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045297, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.61@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.25.60@KRBTEST.COM; defaulting to no policy
I20260806 19:41:37.469141 1538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.439250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34388 (local address 127.10.25.62:43163)
0806 19:41:37.440215 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:37.440224 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:37.442135 (+ 1911us) server_negotiation.cc:407] Connection header received
0806 19:41:37.445124 (+ 2989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:37.445134 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:37.445287 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:37.445443 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:37.448231 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:37.449877 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:37.452956 (+ 3079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:37.453818 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:37.454012 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:37.459771 (+ 5759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:37.459785 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:37.459795 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:37.459901 (+ 106us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:37.464469 (+ 4568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:37.465863 (+ 1394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:37.465868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:37.465871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:37.466020 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:37.466682 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:37.466686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:37.466688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:37.467105 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:37.467229 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:37.468431 (+ 1202us) server_negotiation.cc:299] Negotiation successful
0806 19:41:37.468739 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":429,"threads_started":1}
I20260806 19:41:37.470458 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.440578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43163 (local address 127.0.0.1:34388)
0806 19:41:37.441863 (+ 1285us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.441935 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:37.444691 (+ 2756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:37.446054 (+ 1363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:37.446068 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:37.447095 (+ 1027us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:37.447815 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:37.447841 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:37.450178 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:37.450185 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:37.452712 (+ 2527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:37.452734 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:37.454563 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:37.454572 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:37.454928 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:37.455828 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:37.455864 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:37.459276 (+ 3412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:37.464740 (+ 5464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:37.464754 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:37.464780 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:37.465639 (+ 859us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:37.466184 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:37.466190 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:37.466194 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:37.466469 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:37.467300 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:37.467308 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:37.467808 (+ 500us) client_negotiation.cc:769] Sending connection context
0806 19:41:37.468122 (+ 314us) client_negotiation.cc:240] Negotiation successful
0806 19:41:37.468647 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"thread_start_us":503,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.25.60@KRBTEST.COM" created.
I20260806 19:41:37.473523 1536 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:41:37.484999 1460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } attempt: 5
Aug 06 19:41:37 dist-test-slave-ss4n krb5kdc[1378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045297, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.61@KRBTEST.COM for kudu/127.10.25.61@KRBTEST.COM
Aug 06 19:41:37 dist-test-slave-ss4n krb5kdc[1378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for kudu/127.10.25.61@KRBTEST.COM
I20260806 19:41:37.498456 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.474872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:45701 (local address 127.0.0.1:58550)
0806 19:41:37.475230 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.475262 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:37.475450 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:37.477017 (+ 1567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:37.477027 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:37.477755 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:37.478072 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:37.478091 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:37.479950 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:37.479960 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:37.481853 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:37.481879 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:37.483566 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:37.483576 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:37.483847 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:37.484682 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:37.484714 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:37.488484 (+ 3770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:37.493661 (+ 5177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:37.493674 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:37.493681 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:37.494540 (+ 859us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:37.495173 (+ 633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:37.495183 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:37.495190 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:37.495610 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:37.496670 (+ 1060us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:37.496678 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:37.496951 (+ 273us) client_negotiation.cc:769] Sending connection context
0806 19:41:37.497140 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 19:41:37.497686 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260806 19:41:37.498598 1541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.475214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58550 (local address 127.10.25.61:45701)
0806 19:41:37.476243 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:37.476254 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:37.476299 (+ 45us) server_negotiation.cc:407] Connection header received
0806 19:41:37.476373 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:37.476382 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:37.476669 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:37.476931 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:37.478408 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:37.479696 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:37.482202 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:37.483168 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:37.483504 (+ 336us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:37.488840 (+ 5336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:37.488859 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:37.488876 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:37.488909 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:37.493244 (+ 4335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:37.494746 (+ 1502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:37.494760 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:37.494779 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:37.494953 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:37.495887 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:37.495894 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:37.495897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:37.496468 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:37.496740 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:37.497531 (+ 791us) server_negotiation.cc:299] Negotiation successful
0806 19:41:37.497929 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":599,"threads_started":1}
I20260806 19:41:37.503722 1536 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:41:37.505299 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.504652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53848)
0806 19:41:37.504893 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.505058 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 19:41:37.505443 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.485905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:45701 (local address 127.0.0.1:58554)
0806 19:41:37.486234 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.486256 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:37.486375 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:37.487418 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:37.487426 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:37.487921 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:37.488110 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:37.488120 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:37.490060 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:37.490067 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:37.491690 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:37.491715 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:37.492995 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:37.493003 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:37.493318 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:37.494174 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:37.494194 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:37.497086 (+ 2892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:37.501789 (+ 4703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:37.501796 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:37.501802 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:37.502494 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:37.503128 (+ 634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:37.503138 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:37.503144 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:37.503370 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:37.504118 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:37.504123 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:37.504347 (+ 224us) client_negotiation.cc:769] Sending connection context
0806 19:41:37.504572 (+ 225us) client_negotiation.cc:240] Negotiation successful
0806 19:41:37.505015 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193}
W20260806 19:41:37.505632 1487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
I20260806 19:41:37.506858 1542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.486149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58554 (local address 127.10.25.61:45701)
0806 19:41:37.487104 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:37.487108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:37.487129 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:41:37.487177 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:37.487181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:37.487246 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:37.487376 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:37.488299 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:37.489881 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:37.491903 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:37.492765 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:37.492923 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:37.497532 (+ 4609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:37.497541 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:37.497548 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:37.497585 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:37.501552 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:37.502716 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:37.502721 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:37.502725 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:37.502907 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:37.503549 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:37.503557 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:37.503561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:37.503943 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:37.505832 (+ 1889us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:37.506011 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 19:41:37.506300 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":837,"thread_start_us":620,"threads_started":1}
I20260806 19:41:37.507922 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.507486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53852)
0806 19:41:37.507673 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.507800 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260806 19:41:37.508453 1536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:42241: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
I20260806 19:41:37.508760 1460 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:41:37.510296 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.509777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53858)
0806 19:41:37.510033 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.510127 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 19:41:37.511420 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.511059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53864)
0806 19:41:37.511231 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.511302 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 19:41:37.511677 1460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:42241: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.25.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.60.keytab.
Entry for principal kudu/127.10.25.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.60.keytab.
W20260806 19:41:37.529680 10340 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.10.25.60: real 0.101s user 0.010s sys 0.024s
I20260806 19:41:37.530735 10340 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskduu4iA/build/asan/bin/kudu
/tmp/dist-test-taskduu4iA/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.60:42241
--webserver_interface=127.10.25.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.60.keytab
--principal=kudu/127.10.25.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.25.20:43879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.25.62:43163,127.10.25.61:45701,127.10.25.60:42241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 19:41:37.557966 1536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } attempt: 1
I20260806 19:41:37.559772 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.559022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53874)
0806 19:41:37.559405 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.559561 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190}
I20260806 19:41:37.561785 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.560965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53888)
0806 19:41:37.561244 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.561375 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260806 19:41:37.562299 1536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:42241: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
I20260806 19:41:37.563004 1460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } attempt: 1
I20260806 19:41:37.564435 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.563902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53896)
0806 19:41:37.564138 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.564242 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 19:41:37.565727 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.565290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53906)
0806 19:41:37.565467 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.565602 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260806 19:41:37.565964 1460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:42241: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
I20260806 19:41:37.630688 1536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } attempt: 2
I20260806 19:41:37.632088 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.631577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53916)
0806 19:41:37.631821 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.631918 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 19:41:37.633811 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.633155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53920)
0806 19:41:37.633422 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.633628 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260806 19:41:37.634114 1536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:42241: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
I20260806 19:41:37.634212 1460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } attempt: 2
I20260806 19:41:37.635597 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.635066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53922)
0806 19:41:37.635305 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.635405 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 19:41:37.637072 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.636587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53928)
0806 19:41:37.636825 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.636914 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 19:41:37.637799 1460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:42241: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
I20260806 19:41:37.725445 1536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } attempt: 3
I20260806 19:41:37.727191 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.726572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53930)
0806 19:41:37.726889 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.727002 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260806 19:41:37.729007 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.728223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53946)
0806 19:41:37.728702 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.728827 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
W20260806 19:41:37.729292 1536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:42241: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
I20260806 19:41:37.744233 1460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } attempt: 3
I20260806 19:41:37.746286 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.745368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53962)
0806 19:41:37.745915 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.746056 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315}
I20260806 19:41:37.748498 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.747886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53968)
0806 19:41:37.748213 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.748339 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260806 19:41:37.748795 1460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:42241: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
I20260806 19:41:37.872535 1536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } attempt: 4
I20260806 19:41:37.874193 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.873357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53980)
0806 19:41:37.873802 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.873939 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224}
I20260806 19:41:37.875524 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.875051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53982)
0806 19:41:37.875305 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.875384 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 19:41:37.875761 1536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:42241: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
I20260806 19:41:37.926038 1460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } attempt: 4
I20260806 19:41:37.927543 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.926964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:53998)
0806 19:41:37.927186 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.927292 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 19:41:37.929338 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:37.928900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:54006)
0806 19:41:37.929108 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:37.929193 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260806 19:41:37.930038 1460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.25.60:42241: Network error: Client connection negotiation failed: client connection to 127.10.25.60:42241: connect: Connection refused (error 111)
W20260806 19:41:37.992871 1545 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 19:41:37.993472 1545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 19:41:37.993652 1545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 19:41:37.993829 1545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 19:41:38.003336 1545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 19:41:38.003461 1545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 19:41:38.003576 1545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 19:41:38.003672 1545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 19:41:38.003767 1545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 19:41:38.016788 1545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.25.20:43879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.25.62:43163,127.10.25.61:45701,127.10.25.60:42241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/krb5kdc/kudu_127.10.25.60.keytab
--principal=kudu/127.10.25.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.25.60:42241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.25.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 17:20:36 UTC on ccycloud.quasar-dgoebi.root.comops.site
build id 4570388
ASAN enabled
I20260806 19:41:38.018499 1545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 19:41:38.020150 1545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 19:41:38.036124 1551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:38.036188 1550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 19:41:38.036476 1553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 19:41:38.037931 1545 server_base.cc:1034] running on GCE node
I20260806 19:41:38.039279 1545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 19:41:38.041327 1545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 19:41:38.042765 1545 hybrid_clock.cc:648] HybridClock initialized: now 1786045298042679 us; error 55 us; skew 500 ppm
I20260806 19:41:38.043262 1545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 19:41:38 dist-test-slave-ss4n krb5kdc[1378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045298, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 19:41:38.048026 1545 init.cc:377] Logged in from keytab as kudu/127.10.25.60@KRBTEST.COM (short username kudu)
I20260806 19:41:38.050109 1545 webserver.cc:460] Webserver started at http://127.10.25.60:37313/ using document root <none> and password file <none>
I20260806 19:41:38.051018 1545 fs_manager.cc:359] Metadata directory not provided
I20260806 19:41:38.051126 1545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 19:41:38.051669 1545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 19:41:38.055622 1545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/data/instance:
uuid: "2da60006ea104e5ea1c82fc5ab25b48e"
format_stamp: "Formatted at 2026-08-06 19:41:38 on dist-test-slave-ss4n"
server_key: "dcfc5023e8dd7fb4767e0c478777abfe"
server_key_iv: "23a2eab0a9e0e262799d93a0dced9bfc"
server_key_version: "encryptionkey@0"
I20260806 19:41:38.056702 1545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/wal/instance:
uuid: "2da60006ea104e5ea1c82fc5ab25b48e"
format_stamp: "Formatted at 2026-08-06 19:41:38 on dist-test-slave-ss4n"
server_key: "dcfc5023e8dd7fb4767e0c478777abfe"
server_key_iv: "23a2eab0a9e0e262799d93a0dced9bfc"
server_key_version: "encryptionkey@0"
I20260806 19:41:38.063606 1545 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.007s
I20260806 19:41:38.069276 1560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 19:41:38.071357 1545 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260806 19:41:38.071628 1545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/data,/tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/wal
uuid: "2da60006ea104e5ea1c82fc5ab25b48e"
format_stamp: "Formatted at 2026-08-06 19:41:38 on dist-test-slave-ss4n"
server_key: "dcfc5023e8dd7fb4767e0c478777abfe"
server_key_iv: "23a2eab0a9e0e262799d93a0dced9bfc"
server_key_version: "encryptionkey@0"
I20260806 19:41:38.071864 1545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 19:41:38.096554 1545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 19:41:38.102108 1545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 19:41:38.102487 1545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 19:41:38.125280 1613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.25.60:42241 every 8 connection(s)
I20260806 19:41:38.125705 1545 rpc_server.cc:307] RPC server started. Bound to: 127.10.25.60:42241
I20260806 19:41:38.127496 1545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/data/info.pb
I20260806 19:41:38.132339 1615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 19:41:38.132867 10340 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskduu4iA/build/asan/bin/kudu as pid 1545
I20260806 19:41:38.133082 10340 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskduu4iA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786044861763400-10340-0/minicluster-data/master-2/wal/instance
I20260806 19:41:38.133649 10340 external_mini_cluster.cc:1428] Setting key f6d67a09c2f7559e5c54266dad5d81d4
I20260806 19:41:38.143680 1615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 19:41:38 dist-test-slave-ss4n krb5kdc[1378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.60@KRBTEST.COM
Aug 06 19:41:38 dist-test-slave-ss4n krb5kdc[1378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045298, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.60@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
I20260806 19:41:38.161739 1618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:38.135709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54018 (local address 127.10.25.60:42241)
0806 19:41:38.136802 (+ 1093us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.136815 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.136866 (+ 51us) server_negotiation.cc:407] Connection header received
0806 19:41:38.138770 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.138802 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.139340 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.139863 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.141771 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.143209 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.144757 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.145596 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.146075 (+ 479us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:38.151881 (+ 5806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:38.151900 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:38.151919 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:38.151978 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:38.156432 (+ 4454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.157587 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.157598 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.157609 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.157829 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.158290 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.158297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.158301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.158948 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:38.159322 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:38.159854 (+ 532us) server_negotiation.cc:299] Negotiation successful
0806 19:41:38.160422 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":845,"thread_start_us":499,"threads_started":1}
I20260806 19:41:38.165032 1620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:38.146657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.62:43163 (local address 127.0.0.1:34392)
0806 19:41:38.147903 (+ 1246us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:38.147962 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:38.148149 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:38.148683 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:38.148691 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:38.149426 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:38.149805 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:38.149817 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.151010 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.151020 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:38.152492 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:38.152586 (+ 94us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.153844 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.153850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:38.154031 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:38.154905 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:38.154930 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:38.157844 (+ 2914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:38.161592 (+ 3748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:38.161608 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:38.161620 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:38.162302 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:38.162861 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:38.162867 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:38.162873 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:38.163088 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:38.163717 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:38.163732 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:38.163930 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 19:41:38.164132 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 19:41:38.164531 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":973,"thread_start_us":799,"threads_started":1}
I20260806 19:41:38.165015 1619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:38.146657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34392 (local address 127.10.25.62:43163)
0806 19:41:38.148299 (+ 1642us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.148309 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.148328 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:41:38.148384 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.148388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.148467 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.148606 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.150023 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.150834 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.152845 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.153545 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.153713 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:38.158124 (+ 4411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:38.158131 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:38.158135 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:38.158186 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:38.161365 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.162472 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.162483 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.162488 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.162689 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.163235 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.163241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.163247 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.163565 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:38.163750 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:38.164285 (+ 535us) server_negotiation.cc:299] Negotiation successful
0806 19:41:38.164556 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":774,"thread_start_us":460,"threads_started":1}
I20260806 19:41:38.166767 1615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:41:38.175000 1536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } attempt: 5
Aug 06 19:41:38 dist-test-slave-ss4n krb5kdc[1378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045298, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.60@KRBTEST.COM for kudu/127.10.25.61@KRBTEST.COM
Aug 06 19:41:38 dist-test-slave-ss4n krb5kdc[1378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.62@KRBTEST.COM
Aug 06 19:41:38 dist-test-slave-ss4n krb5kdc[1378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045297, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.61@KRBTEST.COM for kudu/127.10.25.60@KRBTEST.COM
I20260806 19:41:38.186146 1620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:38.167550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.61:45701 (local address 127.0.0.1:58558)
0806 19:41:38.167730 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:38.167772 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:38.167880 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:38.169559 (+ 1679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:38.169565 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:38.170022 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:38.170187 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:38.170196 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.171292 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.171300 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:38.172308 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:38.172321 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.173354 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.173361 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:38.173625 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:38.174250 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:38.174270 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:38.178188 (+ 3918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:38.183093 (+ 4905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:38.183100 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:38.183105 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:38.183577 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:38.184037 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:38.184043 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:38.184047 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:38.184218 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:38.184978 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:38.184984 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:38.185174 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 19:41:38.185339 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 19:41:38.185716 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 19:41:38.186201 1621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:38.168068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58558 (local address 127.10.25.61:45701)
0806 19:41:38.168909 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.168916 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.168938 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:41:38.169009 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.169013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.169085 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.169515 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.170352 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.171138 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.172482 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.173169 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.173333 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:38.178452 (+ 5119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:38.178460 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:38.178465 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:38.178658 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:38.182935 (+ 4277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.183716 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.183723 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.183728 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.183900 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.184345 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.184352 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.184355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.184805 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:38.185006 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:38.185452 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 19:41:38.185726 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":549,"threads_started":1}
I20260806 19:41:38.187971 1619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:38.165813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34406 (local address 127.10.25.62:43163)
0806 19:41:38.166039 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.166046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.166067 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:41:38.166125 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.166129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.166206 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.166331 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.168932 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.169990 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.171567 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.172063 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.172224 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:38.178795 (+ 6571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:38.178801 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:38.178805 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:38.178879 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:38.182978 (+ 4099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.184138 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.184146 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.184150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.184332 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.184824 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.184833 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.184839 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.185268 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:38.185644 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:38.187358 (+ 1714us) server_negotiation.cc:299] Negotiation successful
0806 19:41:38.187593 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 19:41:38.188022 1615 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } has no permanent_uuid. Determining permanent_uuid...
I20260806 19:41:38.195482 1539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:38.175677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:54026)
0806 19:41:38.175886 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:38.175908 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:38.176029 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:38.176716 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:38.176721 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:38.177171 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:38.177430 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:38.177447 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.179015 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.179022 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:38.180805 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:38.180830 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.182237 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.182247 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:38.182574 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:38.183324 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:38.183356 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:38.186738 (+ 3382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:38.191957 (+ 5219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:38.191964 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:38.191969 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:38.192631 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:38.193111 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:38.193118 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:38.193122 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:38.193324 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:38.194215 (+ 891us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:38.194221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:38.194435 (+ 214us) client_negotiation.cc:769] Sending connection context
0806 19:41:38.194587 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 19:41:38.194950 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 19:41:38.196915 1618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:38.176239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54026 (local address 127.10.25.60:42241)
0806 19:41:38.176417 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.176421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.176437 (+ 16us) server_negotiation.cc:407] Connection header received
0806 19:41:38.176482 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.176486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.176561 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.176682 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.177665 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.178751 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.181007 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.182015 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.182224 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:38.186982 (+ 4758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:38.186992 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:38.187000 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:38.187051 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:38.191767 (+ 4716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.192782 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.192788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.192792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.192953 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.193475 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.193510 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.193515 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.194034 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:38.196004 (+ 1970us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:38.196161 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 19:41:38.196481 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":65}
Aug 06 19:41:38 dist-test-slave-ss4n krb5kdc[1378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045298, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.60@KRBTEST.COM for kudu/127.10.25.60@KRBTEST.COM
Aug 06 19:41:38 dist-test-slave-ss4n krb5kdc[1378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.25.61@KRBTEST.COM
I20260806 19:41:38.204514 1536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9: Bootstrap starting.
I20260806 19:41:38.207330 1460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } attempt: 5
I20260806 19:41:38.207645 1620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:38.189000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:54040)
0806 19:41:38.189203 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:38.189226 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:38.189341 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:38.191116 (+ 1775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:38.191129 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:38.191888 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:38.192109 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:38.192119 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.193741 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.193748 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:38.195303 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:38.195321 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.196431 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.196438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:38.196650 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:38.197435 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:38.197454 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:38.200070 (+ 2616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:38.204200 (+ 4130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:38.204208 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:38.204214 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:38.204850 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:38.205351 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:38.205356 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:38.205359 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:38.205588 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:38.206291 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:38.206296 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:38.206470 (+ 174us) client_negotiation.cc:769] Sending connection context
0806 19:41:38.206657 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 19:41:38.207066 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 19:41:38.209231 1622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:38.189385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54040 (local address 127.10.25.60:42241)
0806 19:41:38.190780 (+ 1395us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.190784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.190804 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:41:38.190848 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.190851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.190907 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.191307 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.192297 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.193582 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.195522 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.196227 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.196982 (+ 755us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:38.200317 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:38.200323 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:38.200327 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:38.200360 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:38.204029 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.205037 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.205042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.205046 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.205221 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.205733 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.205738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.205741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.206106 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:38.208297 (+ 2191us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:38.208464 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 19:41:38.208716 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1279,"thread_start_us":1079,"threads_started":1}
I20260806 19:41:38.211073 1536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:38.211213 1621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:38.190189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58566 (local address 127.10.25.61:45701)
0806 19:41:38.190654 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.190662 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.190683 (+ 21us) server_negotiation.cc:407] Connection header received
0806 19:41:38.190746 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.190753 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.190845 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.190962 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.192930 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.193997 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.195387 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.196072 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.196237 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:38.202954 (+ 6717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:38.202962 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:38.202967 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:38.203015 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:38.206546 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.207540 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.207546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.207549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.207736 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.208337 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.208345 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.208352 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.208788 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:38.209106 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:38.210523 (+ 1417us) server_negotiation.cc:299] Negotiation successful
0806 19:41:38.210816 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318}
I20260806 19:41:38.212690 1536 log.cc:824] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9: Log is configured to *not* fsync() on all Append() calls
Aug 06 19:41:38 dist-test-slave-ss4n krb5kdc[1378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786045296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.25.62@KRBTEST.COM for kudu/127.10.25.60@KRBTEST.COM
I20260806 19:41:38.220181 1615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e: Bootstrap starting.
I20260806 19:41:38.220271 10340 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 19:41:38.222975 1536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9: No bootstrap required, opened a new log
I20260806 19:41:38.225277 1615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:38.227011 1615 log.cc:824] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:38.224877 1434 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "bdb659eba3eb4e28b278c77ea3589434"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34406
I20260806 19:41:38.231896 1615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e: No bootstrap required, opened a new log
I20260806 19:41:38.232352 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:38.208101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.25.60:42241 (local address 127.0.0.1:54052)
0806 19:41:38.208571 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 19:41:38.208592 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 19:41:38.208707 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 19:41:38.209360 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 19:41:38.209368 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 19:41:38.209967 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0806 19:41:38.210214 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:38.210230 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.212038 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.212044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:38.213102 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 19:41:38.213117 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.214362 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.214369 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 19:41:38.214576 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:38.215374 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 19:41:38.215417 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 19:41:38.218394 (+ 2977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 19:41:38.223761 (+ 5367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:38.223770 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:38.223775 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:38.224437 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:38.229988 (+ 5551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 19:41:38.229995 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 19:41:38.229999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 19:41:38.230298 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 19:41:38.231159 (+ 861us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 19:41:38.231166 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 19:41:38.231368 (+ 202us) client_negotiation.cc:769] Sending connection context
0806 19:41:38.231554 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 19:41:38.231928 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300}
I20260806 19:41:38.232911 1618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:38.208316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54052 (local address 127.10.25.60:42241)
0806 19:41:38.208981 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.208988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.209007 (+ 19us) server_negotiation.cc:407] Connection header received
0806 19:41:38.209062 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.209068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.209151 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.210402 (+ 1251us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.210577 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.211803 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.213278 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.214158 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.218625 (+ 4467us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:38.218939 (+ 314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:38.218946 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:38.218952 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:38.218995 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:38.223514 (+ 4519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.229572 (+ 6058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.229583 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.229588 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.229818 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.230488 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.230494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.230498 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.230972 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:38.231206 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:38.231740 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 19:41:38.232171 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500}
I20260806 19:41:38.235452 1536 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } }
I20260806 19:41:38.235947 1536 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c7b0ebc590426589c7a2d54f081de9, State: Initialized, Role: FOLLOWER
I20260806 19:41:38.236598 1536 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } }
I20260806 19:41:38.238157 1625 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } } }
I20260806 19:41:38.239194 1625 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:41:38.239548 1536 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:38.240069 1460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434: Bootstrap starting.
I20260806 19:41:38.245419 1615 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } }
I20260806 19:41:38.246074 1615 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2da60006ea104e5ea1c82fc5ab25b48e, State: Initialized, Role: FOLLOWER
I20260806 19:41:38.246110 1460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434: Neither blocks nor log segments found. Creating new log.
I20260806 19:41:38.246730 1615 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } }
I20260806 19:41:38.248128 1460 log.cc:824] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434: Log is configured to *not* fsync() on all Append() calls
I20260806 19:41:38.248401 1629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } } }
I20260806 19:41:38.250032 1629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:41:38.250314 1615 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:38.255702 1460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434: No bootstrap required, opened a new log
I20260806 19:41:38.267035 1460 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } }
I20260806 19:41:38.267536 1460 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdb659eba3eb4e28b278c77ea3589434, State: Initialized, Role: FOLLOWER
I20260806 19:41:38.268456 1460 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } }
W20260806 19:41:38.283175 1647 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 19:41:38.282922 1644 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } } }
I20260806 19:41:38.284053 1644 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:41:38.284328 1460 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 19:41:38.284409 1434 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 19:41:38.284669 1434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [term 0 FOLLOWER]: Advancing to term 1
W20260806 19:41:38.285581 1647 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 19:41:38.288700 1434 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } }
I20260806 19:41:38.291196 1434 leader_election.cc:288] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [CANDIDATE]: Term 1 election: Requested vote from peers f3c7b0ebc590426589c7a2d54f081de9 (127.10.25.61:45701), 2da60006ea104e5ea1c82fc5ab25b48e (127.10.25.60:42241)
I20260806 19:41:38.291967 1511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bdb659eba3eb4e28b278c77ea3589434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f3c7b0ebc590426589c7a2d54f081de9"
I20260806 19:41:38.292183 1589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bdb659eba3eb4e28b278c77ea3589434" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2da60006ea104e5ea1c82fc5ab25b48e"
I20260806 19:41:38.292665 1511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:38.293044 1589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e [term 0 FOLLOWER]: Advancing to term 1
I20260806 19:41:38.298005 1511 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdb659eba3eb4e28b278c77ea3589434 in term 1.
I20260806 19:41:38.298870 1589 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdb659eba3eb4e28b278c77ea3589434 in term 1.
I20260806 19:41:38.299115 1408 leader_election.cc:302] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdb659eba3eb4e28b278c77ea3589434, f3c7b0ebc590426589c7a2d54f081de9; no voters:
I20260806 19:41:38.299800 1644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 19:41:38.303862 1653 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 19:41:38.277609 1467 master_runner.cc:186] Error getting master registration for 127.10.25.62:43163: OK, instance_id { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" instance_seqno: 1786045296754323 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 19:41:38.326704 1467 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 19:41:38.329660 1644 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [term 1 LEADER]: Becoming Leader. State: Replica: bdb659eba3eb4e28b278c77ea3589434, State: Running, Role: LEADER
I20260806 19:41:38.330534 1644 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } }
I20260806 19:41:38.309387 1545 master_runner.cc:186] Error getting master registration for 127.10.25.62:43163: OK, instance_id { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" instance_seqno: 1786045296754323 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 19:41:38.336081 1545 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 19:41:38.340412 1655 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdb659eba3eb4e28b278c77ea3589434. Latest consensus state: current_term: 1 leader_uuid: "bdb659eba3eb4e28b278c77ea3589434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } } }
I20260806 19:41:38.341085 1655 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:38.347177 1668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260806 19:41:38.355064 1567 tablet.cc:2369] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:41:38.370117 1668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 19:41:38.384721 1618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:38.348477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54054 (local address 127.10.25.60:42241)
0806 19:41:38.348719 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.348728 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.356317 (+ 7589us) server_negotiation.cc:407] Connection header received
0806 19:41:38.356407 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.356415 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.356535 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.356648 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.364596 (+ 7948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.365927 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.368973 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.369881 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.370019 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:38.374658 (+ 4639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:38.374666 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:38.374671 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:38.374727 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:38.379386 (+ 4659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.380653 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.380665 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.380668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.381117 (+ 449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.382049 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.382057 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.382060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.382659 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:38.382992 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:38.383751 (+ 759us) server_negotiation.cc:299] Negotiation successful
0806 19:41:38.384289 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 19:41:38.391736 1489 tablet.cc:2369] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 19:41:38.392107 1619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:38.339866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34420 (local address 127.10.25.62:43163)
0806 19:41:38.340094 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.340100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.356195 (+ 16095us) server_negotiation.cc:407] Connection header received
0806 19:41:38.356489 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.356496 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.356588 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.356681 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.359496 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.360887 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.365200 (+ 4313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.366031 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.366151 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:38.380298 (+ 14147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:38.380306 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:38.380312 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:38.380365 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:38.386673 (+ 6308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.388564 (+ 1891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.388570 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.388573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.388743 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.389615 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.389620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.389624 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.390065 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:38.390353 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:38.391400 (+ 1047us) server_negotiation.cc:299] Negotiation successful
0806 19:41:38.391693 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 19:41:38.394425 1674 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 19:41:38.397310 1589 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e [term 1 FOLLOWER]: Refusing update from remote peer bdb659eba3eb4e28b278c77ea3589434: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:41:38.397593 1511 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9 [term 1 FOLLOWER]: Refusing update from remote peer bdb659eba3eb4e28b278c77ea3589434: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 19:41:38.398520 1655 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:41:38.398872 1644 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 19:41:38.400741 1621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:38.354839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58582 (local address 127.10.25.61:45701)
0806 19:41:38.356234 (+ 1395us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.356242 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.367418 (+ 11176us) server_negotiation.cc:407] Connection header received
0806 19:41:38.367780 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.367789 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.367902 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.367983 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.370670 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.371950 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.375209 (+ 3259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:38.376044 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:38.376173 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:38.388307 (+ 12134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 19:41:38.388317 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 19:41:38.388323 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 19:41:38.388375 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 19:41:38.395788 (+ 7413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.396867 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.396873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.396876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.397108 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 19:41:38.397778 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 19:41:38.397783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 19:41:38.397786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 19:41:38.398258 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 19:41:38.399844 (+ 1586us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:38.399979 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 19:41:38.400281 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1255}
I20260806 19:41:38.432375 1625 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdb659eba3eb4e28b278c77ea3589434. Latest consensus state: current_term: 1 leader_uuid: "bdb659eba3eb4e28b278c77ea3589434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } } }
I20260806 19:41:38.432832 1625 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:41:38.436492 1629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdb659eba3eb4e28b278c77ea3589434. Latest consensus state: current_term: 1 leader_uuid: "bdb659eba3eb4e28b278c77ea3589434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } } }
I20260806 19:41:38.436918 1629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:41:38.442045 1675 mvcc.cc:205] Tried to move back new op lower bound from 7315641542199328768 to 7315641541970395136. Current Snapshot: MvccSnapshot[applied={T|T < 7315641542199328768}]
I20260806 19:41:38.454339 1677 mvcc.cc:205] Tried to move back new op lower bound from 7315641542199328768 to 7315641541970395136. Current Snapshot: MvccSnapshot[applied={T|T < 7315641542199328768}]
I20260806 19:41:38.467880 1625 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bdb659eba3eb4e28b278c77ea3589434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } } }
I20260806 19:41:38.468248 1625 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:41:38.480562 1629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bdb659eba3eb4e28b278c77ea3589434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } } }
I20260806 19:41:38.480885 1629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 19:41:38.483170 1655 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bdb659eba3eb4e28b278c77ea3589434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } } }
I20260806 19:41:38.483525 1655 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:38.484208 1655 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bdb659eba3eb4e28b278c77ea3589434" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb659eba3eb4e28b278c77ea3589434" member_type: VOTER last_known_addr { host: "127.10.25.62" port: 43163 } } peers { permanent_uuid: "f3c7b0ebc590426589c7a2d54f081de9" member_type: VOTER last_known_addr { host: "127.10.25.61" port: 45701 } } peers { permanent_uuid: "2da60006ea104e5ea1c82fc5ab25b48e" member_type: VOTER last_known_addr { host: "127.10.25.60" port: 42241 } } }
I20260806 19:41:38.484506 1655 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434 [sys.catalog]: This master's current role is: LEADER
I20260806 19:41:38.488327 1668 catalog_manager.cc:1347] Generated new cluster ID: 32d21af8246e486bb7087126acdea391
I20260806 19:41:38.488466 1668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 19:41:38.515949 1668 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 19:41:38.518290 1668 catalog_manager.cc:1504] Loading token signing keys...
I20260806 19:41:38.536823 1668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdb659eba3eb4e28b278c77ea3589434: Generated new TSK 0
I20260806 19:41:38.538363 1668 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 19:41:38.652952 1619 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:38.642744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34428 (local address 127.10.25.62:43163)
0806 19:41:38.642974 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.642982 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.650324 (+ 7342us) server_negotiation.cc:407] Connection header received
0806 19:41:38.651015 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.651024 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.651123 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.651202 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.652671 (+ 1469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34428: BlockingRecv error: recv got EOF from 127.0.0.1:34428 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260806 19:41:38.656059 1618 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:38.642284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54058 (local address 127.10.25.60:42241)
0806 19:41:38.642508 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.642515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.654054 (+ 11539us) server_negotiation.cc:407] Connection header received
0806 19:41:38.654385 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.654391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.654487 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.654570 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.655746 (+ 1176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54058: BlockingRecv error: recv got EOF from 127.0.0.1:54058 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260806 19:41:38.656390 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:38.654328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58588 (local address 127.10.25.61:45701)
0806 19:41:38.654611 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.654618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.654640 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:41:38.654840 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.654846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.654925 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.654998 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.656141 (+ 1143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58588: BlockingRecv error: recv got EOF from 127.0.0.1:58588 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 19:41:38.666644 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:38.663334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58594 (local address 127.10.25.61:45701)
0806 19:41:38.663600 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.663606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.664271 (+ 665us) server_negotiation.cc:407] Connection header received
0806 19:41:38.664686 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.664693 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.664838 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.664934 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.666311 (+ 1377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58594: BlockingRecv error: recv got EOF from 127.0.0.1:58594 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260806 19:41:38.676236 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:38.673272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58600 (local address 127.10.25.61:45701)
0806 19:41:38.673531 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.673538 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.674262 (+ 724us) server_negotiation.cc:407] Connection header received
0806 19:41:38.674603 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.674610 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.674715 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.674797 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.675973 (+ 1176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58600: BlockingRecv error: recv got EOF from 127.0.0.1:58600 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260806 19:41:38.689986 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:38.687237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58612 (local address 127.10.25.61:45701)
0806 19:41:38.687450 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.687458 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.688106 (+ 648us) server_negotiation.cc:407] Connection header received
0806 19:41:38.688358 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.688365 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.688484 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.688604 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.689676 (+ 1072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58612: BlockingRecv error: recv got EOF from 127.0.0.1:58612 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260806 19:41:38.702215 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:38.699115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58622 (local address 127.10.25.61:45701)
0806 19:41:38.699426 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.699435 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.700197 (+ 762us) server_negotiation.cc:407] Connection header received
0806 19:41:38.700471 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.700479 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.700608 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.700772 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.701883 (+ 1111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58622: BlockingRecv error: recv got EOF from 127.0.0.1:58622 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260806 19:41:38.717051 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:38.714329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58638 (local address 127.10.25.61:45701)
0806 19:41:38.714531 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.714538 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.715054 (+ 516us) server_negotiation.cc:407] Connection header received
0806 19:41:38.715277 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.715284 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.715384 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.715496 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.716697 (+ 1201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58638: BlockingRecv error: recv got EOF from 127.0.0.1:58638 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 19:41:38.741158 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:38.737954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58644 (local address 127.10.25.61:45701)
0806 19:41:38.738231 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.738238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.739068 (+ 830us) server_negotiation.cc:407] Connection header received
0806 19:41:38.739337 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.739347 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.739482 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.739628 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.740828 (+ 1200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58644: BlockingRecv error: recv got EOF from 127.0.0.1:58644 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
W20260806 19:41:38.797665 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:38.795030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58648 (local address 127.10.25.61:45701)
0806 19:41:38.795271 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.795277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.795847 (+ 570us) server_negotiation.cc:407] Connection header received
0806 19:41:38.796119 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.796126 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.796267 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.796401 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.797348 (+ 947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58648: BlockingRecv error: recv got EOF from 127.0.0.1:58648 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260806 19:41:38.871328 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:38.868403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58650 (local address 127.10.25.61:45701)
0806 19:41:38.868639 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:38.868646 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:38.869387 (+ 741us) server_negotiation.cc:407] Connection header received
0806 19:41:38.869652 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:38.869659 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:38.869803 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:38.869938 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:38.871051 (+ 1113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58650: BlockingRecv error: recv got EOF from 127.0.0.1:58650 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260806 19:41:39.006408 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:39.004280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58656 (local address 127.10.25.61:45701)
0806 19:41:39.004561 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:39.004568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:39.004750 (+ 182us) server_negotiation.cc:407] Connection header received
0806 19:41:39.005017 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:39.005030 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:39.005107 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:39.005236 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:39.006115 (+ 879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58656: BlockingRecv error: recv got EOF from 127.0.0.1:58656 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
W20260806 19:41:39.270952 1621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 19:41:39.268323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58664 (local address 127.10.25.61:45701)
0806 19:41:39.268603 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:39.268611 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:39.268927 (+ 316us) server_negotiation.cc:407] Connection header received
0806 19:41:39.269245 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:39.269259 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:39.269393 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:39.269577 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0806 19:41:39.270608 (+ 1031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58664: BlockingRecv error: recv got EOF from 127.0.0.1:58664 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
I20260806 19:41:39.288228 1647 catalog_manager.cc:1259] Loaded cluster ID: 32d21af8246e486bb7087126acdea391
I20260806 19:41:39.288353 1647 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9: loading cluster ID for follower catalog manager: success
I20260806 19:41:39.291353 1647 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9: acquiring CA information for follower catalog manager: success
I20260806 19:41:39.293612 1647 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f3c7b0ebc590426589c7a2d54f081de9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 19:41:39.310169 1653 catalog_manager.cc:1259] Loaded cluster ID: 32d21af8246e486bb7087126acdea391
I20260806 19:41:39.310338 1653 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e: loading cluster ID for follower catalog manager: success
I20260806 19:41:39.313226 1653 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e: acquiring CA information for follower catalog manager: success
I20260806 19:41:39.315562 1653 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2da60006ea104e5ea1c82fc5ab25b48e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 19:41:39.795864 1749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:39.788323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58666 (local address 127.10.25.61:45701)
0806 19:41:39.788847 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:39.788852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:39.789210 (+ 358us) server_negotiation.cc:407] Connection header received
0806 19:41:39.789393 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:39.789399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:39.789799 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:39.789993 (+ 194us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:41:39.790592 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:39.791415 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:39.793059 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:39.793775 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:39.793938 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:39.794584 (+ 646us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:41:39.795219 (+ 635us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:41:39.795356 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:39.795494 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 19:41:39.795589 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":305,"threads_started":1}
I20260806 19:41:39.810364 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:39.802935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54074 (local address 127.10.25.60:42241)
0806 19:41:39.803749 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:39.803754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:39.803774 (+ 20us) server_negotiation.cc:407] Connection header received
0806 19:41:39.804093 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:39.804097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:39.804271 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:39.804444 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:41:39.805143 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:39.806307 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:39.807732 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:39.808242 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:39.808366 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:39.808958 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:41:39.809634 (+ 676us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:41:39.809773 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:39.809961 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 19:41:39.810046 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":455,"threads_started":1}
I20260806 19:41:39.827814 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:39.818244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54076 (local address 127.10.25.60:42241)
0806 19:41:39.818512 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:39.818523 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:39.819895 (+ 1372us) server_negotiation.cc:407] Connection header received
0806 19:41:39.820458 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:39.820472 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:39.820611 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:39.820740 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:41:39.821711 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:39.823071 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:39.825836 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:39.826287 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:39.826443 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:39.826996 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:41:39.827164 (+ 168us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:41:39.827277 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:39.827399 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 19:41:39.827506 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 19:41:39.828465 1749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:39.818768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58674 (local address 127.10.25.61:45701)
0806 19:41:39.818970 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:39.818974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:39.820897 (+ 1923us) server_negotiation.cc:407] Connection header received
0806 19:41:39.821259 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:39.821273 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:39.821349 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:39.821470 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:41:39.822275 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:39.823479 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:39.825884 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:39.826625 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:39.826782 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:39.827256 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:41:39.827404 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:41:39.827657 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:39.828132 (+ 475us) server_negotiation.cc:299] Negotiation successful
0806 19:41:39.828237 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 19:41:39.829154 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 19:41:39.817887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34440 (local address 127.10.25.62:43163)
0806 19:41:39.820195 (+ 2308us) server_negotiation.cc:206] Beginning negotiation
0806 19:41:39.820204 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 19:41:39.820226 (+ 22us) server_negotiation.cc:407] Connection header received
0806 19:41:39.820373 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 19:41:39.820381 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 19:41:39.820659 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 19:41:39.820872 (+ 213us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 19:41:39.821785 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:39.822825 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:39.825886 (+ 3061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 19:41:39.826632 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 19:41:39.827332 (+ 700us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 19:41:39.827563 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 19:41:39.828085 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 19:41:39.828674 (+ 589us) server_negotiation.cc:1035] Waiting for connection context
0806 19:41:39.828730 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 19:41:39.828845 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2083,"thread_start_us":2015,"threads_started":1}
I20260806 19:41:39.835067 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 1391
I20260806 19:41:39.836320 1455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:39.960834 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 1391
W20260806 19:41:39.989993 1563 connection.cc:535] server connection from 127.0.0.1:54052 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 19:41:39.990445 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 1467
I20260806 19:41:39.991601 1531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:40.112983 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 1467
I20260806 19:41:40.141319 10340 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskduu4iA/build/asan/bin/kudu pid 1545
I20260806 19:41:40.143005 1609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 19:41:40.262634 10340 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskduu4iA/build/asan/bin/kudu with pid 1545
2026-08-06T19:41:40Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5738 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (11144 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (438417 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST